“POCUBENT RESUME 
. | 


‘ED 185 765 ee . & RC 123 920 


‘ : 7 
AUTHOR | rssalayke/ James E., Ed.: Mirkin, Phyllis K., Ed. 
TITLE Proceed s of the Minnesota Roundtable Conference on 
Assessment of Learning Disabled: Children. . 
- INSTITUTION Minnesota Univ.,- Minneapolis. Inst. for. necceren on -. 
aa ' ' Learning: Disabilities. 

SPONS AGENCY Bureau of Fducation ‘for the Handicapped: (DHEW/OE) 

: Washingtor, D.c. ~~. } 

REPORT NO  ~ IRLD-Mono-8 . oS 

PUB DATE ~ Apr 79 - . a ; ‘ 

CONTRACT - 300-77-0491 ee . of | . 
‘NOTE 4159p.3 See’ also EC 123° "901-925. A, “— 
; - tes 

EDRS PRICE ' MFOT/PCO7 Plus Postage. 


DESCRIPTORS “™ *Decision Making: Elementary eeeondacs Ed ugation; 
Evaluation Methods’ Intervention; *Learning 


= . ¢ Disabilities: «Program Evaluation; *Stude nt so? 
 -'. Ewaluation: Student Placement | , 
ABSTRACT a ’ 
i‘ ’ Proeeédin ds from the Minnesota Roundtable Conference 


on Assessment of Learning Disabled children include two major 
presentations reviewing research and re ctions) to those presentations 
from conference participants. J. Ysseldyke's presentation . 3 
"Psychoeducational~ Assessment and Decision Making" reviews basic 
considerations underlying his research on the’ assessment- -intervention 
process; describes’ some of the issues involved (such as definitional 
debates,’ use of tests. for purposes other than thase for which =aet 
were intended, and questions reaarding the tests' technical © 
adequacy) : and outlines ‘six areas of tesearch (including computee 
simulation studies and ecological research on placerent team decision 
making). Reactions to the paper are given by B. Keogh, L. Goodman, 
and R. Woodcock. Discussion hiohlights on the topic fellow. In the 

alien major presentation, "Behavioral Research Methodclogy as a 

, asis for the.Formatjve Evaluation of Learning Disability Services," 
S. Deno outlines bac gréund factors and assumptions of a 3 year study 
en the effectiveness of teacher implemented systematic formative 

uation. T. Lovitt, P. Newcomer, and J. Jenkins pfrevide reactions, 

with h\ highlights of the discussion included. A Summary of discussion 
comments concluding. the conference is also provided. (CL) 


A 


\ 


~ 


e 
Je III IOI IOI IE IOI JOSIE IOI IOI OI IDI OISIOIIOIDI IOI IOI IOIOIIDII III I III IDI IOI TOI Ik 


* Reproductions supplied by FDRS Are the best that can be made * 


* ’ from the original document. * 
SEIS IORI III IOI IOI IOI DI IOI IOIOIIS ISI IDI CII IOI TOIT IIIT ATT T IIIA TIT SII IA A IA 


‘ 


ERIC ee a oe et 


wy 


“PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED 


James Ysseldyke 


2 ae : e 
, 


TO THE EDUCATIONAL Rt SOURCES 
INFORMATION GEN TER (ERIC) 


US DEPARTMENT OF HEALTH, 
EDUCATION & WELFARE 
NATIONAL INSTITUTE OF 

EDUCATION 


THIS DOCUMENT HAS BEEN REPRO- 


DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIGIN. 
ATING IT POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRE- 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUIGATION POSITION OR POLICY 


‘Director; James —.*Ysseldyke 7 i oar 


tigktors are conducting research on the assessment /decision-making/ ? 
intervention process as it relates:to learning disabled chilgrgn s ‘ 
. Research activities are organized HHES eight © major areas: eae oy 7 , 


’ Dectstoh-making/ Intervention 


% 
> 


Associate Director: Phyllis K. Mirkin se ek <. 9 
. ls : a : a7 ’ 


\ : , .* 


a contract (300-77-0491) with the Bureau of Education for 
capped, Department of Health,: Education, and ‘Welfare, U.S.. Qffice of . _ 
Education, through Title VI-G of Public Law. 91-230. Institute inves- ~ Mey ; 


eT, equa of ‘Norm-Referenced Date for Prediction © Si 7 


oY 


of oHeeeee 
- @° TTS Computer Simulation aaaates Sinha Assessment/ | _ 
ocess 
Loe ra cae . , . 
WD and * | . 
Labeled LD «' > # 


gonparative Research on Children Lab 
a Academically hut 


IV. Surveys on ‘In-the-Fleld Asses ment, ketsion Waking, : 
) \and Intervention , 


4 


oe 
‘ 


Vv. Ethological Research on Placement‘Team Decision 
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James E. Ysseldyke : a: 
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whe Institute for Research on Learning Disabilities at the Univer- ms 


s 


sity of tieneaeka is one of. five Institutes funded by the Division of a 


% 


. Innovation and Development, Bureau of Education for the Handicapped. 


The Institute is focusing its research activities on critical and com- 


. 


plex theoretical and empirical (esues in the assessment-intervent Lon} 

and ee seienaktac processes. throughout the year ocean the Con- 
ference,’ Institute staff reviewed both the knowledge base and the state aa 
of the art in assessment and decision making, and outlined a plan to 

guide research activities during the next two years. 


The Roundtable Conference afforded an ppportunity for individuals 
’ _ . -{ a 
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with varied backgrounds in research, instruction, servite, and test con- 
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, | ~ t 
struction to react to what had already been done as eis as to what was ee 


planned for the future; to shaté current perspectives on the state of, 


os 


the art and to propose future directions in research on the assessment 


of learning disabled children. The aim of the Conference was to create - ‘ 


? 


an atmosphere in which participants would be free to armchaifr-to - te. ge 


speculate beyond current data and of fer intuitive hunches about how : a ‘ 


best to address the a needs in this area--to address’ issues ih - | 


the context both of what has been and what might be. 


a Hill, .the site of the Conference, was an apprepe tate en- 


gn for exchange, Speculations and evalyation of ideas and views. — 
4 - ~ 
a 
_ The partictpants sitting at the Roundtable were persons who shared the sO 8 
( eae ¢ . q , ‘ : 
Institute's serious investment in-the improvement of assessihent, : \ 


decision making, and intervention for learning cfsabted Snr hanenis: 


“ 


who eDetteved that through a cooperative, collaborative Stfoxe, giant 


4 
strides could be made in that direction. ‘ 


“! ! 
Unlike many other conferences, an audience was invited fo eaves- 


drop and observe Roundtable exchanges, but more importartly to share 


ideas and eeackions with ‘he presenters during the two: days of the 


interes: Participants WEnC urged to extend the Roundtable spirit 


‘ ‘ ' s ' . 
beyond the conference room, to continue to communicate thoughts and 


concerns at coffee breaks,> lunch, and seks, 


‘Two sessions on the first day, of the Conference each began with 


. 
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_. : ai . , , ay 
one-hour presentations that reviewed the focal issues of the research 


and outlined, in general, the proposed. activities of the research tedmst 


ry 
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The presentations derived directly from the previous year's efforts to 
review the knowledge base for current practices and to outline research - 


believed necessary for addressing specific complex issues.+ Following 


» 


each presentation, three individuals reacted to both the reviews and 
! : : 7 
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the research plans. Reactants were asked to address’ three issues: 


(1) the extent to which the principal investigators. and their collaborar 


tors had adequately reviewed the current jmowledge base frém each of 
their particular perspectives, (2) MRD eene to which the rationale 
for, doing regearch fn a particular area was adequately a ane 
@). the potential payoff for the field, and learning disabled ere 
in’ particular, of fie. planned research. 

Following these presentations, Individuals at the Roundtable 
were provided an opportunity to question the speakers for clarification 


~ Be . 
SE 
‘Sr elaboration, comment on specific issues of concern to them, and . 


to dikcuss among each okher those issues and concerns,. They were er 


able to; as it wére, speculate, debate, and evaluate. After the 
a % - i < 


Roundtable discussion, members of the Har ticipant-observer audience ¢ 


x . " 
ny a = 


were given an opportunity ‘to raise issues of concern to them and to- + - ie 


. ask questions of specific speakers. 


» t 
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The ,second day:of the Conference began with comments by John 


‘ ¥ . : Py 


Guthrie and John Salvia, who were asked,to make some summarizing, ~ 


«: 


bd ‘ 


integrative comments regarding the first day's discussions, as well 


* 
° as to offer some thoughts regarding overall direction in this 4rea 
: of ‘pheeanch “Their remarks were followed by discussion among those at 
. : foe 3 . 
the Roundtable and, later, by comments and questions. from the audience ° 
; — - ace - ; pare 
of participant-observers. ‘ 
._ ° ‘The Roundtable Confetence and the preparation. of this monograph . 
oe é rea 4 . 
were sponsored by funds made available’ from) the Division of Innovation. - ® 
: i i . . : B) 
and Development, Buneee of Education for the Handicapped. The FoLtoe: 
, ing are the fadtvaduatls who participated in the Roundtable Conference: LN 
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: : : ; . Maynaxgd C. Reynolds ¢ 
, University of Minnesota J 


This 4g ah extraordinary, perfod in Special Education. We are 
~ e « . a 
, ‘ ‘ \ . a oS . 4 
at tHe mountain top inthe history of Special Education, at least, +” 
in level-of activity. This is the year in which the schools are 


mandated literally to, locate and evaluate. every handicapped child. 


mS 


} It ts the year in which individdalized flans, millions of them, 
: ‘i : 


| . ’ : 
; have had to be written - one hy pne - for all of these children. 
rs . “ : ; 

. & 
‘Truly, we have a great deal on the educational plate in 1978 as : 


. 


concerns handicapped students! 


. , Events are running ‘rapidly; in the movement, of children from 


i \ ae 
- i a to another, in the increasing Studies,of individual ( 
€ Fi 
‘ children, in after-school training sessions for teachers, and much 


more. There is a-great Press to get on with things very rapidly; 


legal rights are present rights. The procedural demands of new laws 
and regulations are very great. | 
. : d 
The responses to all of these changes in many schod1 meee? ae 
Z and ia te uriiversities as well have been quite seaaue iat: Most | 


: x i 7 ‘ - 1 
~ teachers have not been well prepared to write their pres Colleges 


have not geared themselves up to be. helpful in the retraining pro- 
cesses. Some “school leaders hope that most of the new activity will 


prove to’ be just another fad and disappear soon; and there. is some 
— ‘ é 4 ‘ ‘ . 
: Yr. 
downright hostility to the whole set of changes. Predictably, there 
= ' will be attempts to take apart some of the legislation and regulations , 
: ~ oo : . rated 


_ ‘which now direct programs for the handicapped. ~ 


ie ~~ w 
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A Individualized Educational Programs - required for all handicapped aa 
students ‘under Public Law 94-142. we 
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“It is teworthy that all of @he busy activities of the year are 


J going on ina ch mpre public way than ever before, so that where there 


{ is incompetency thete is no hiding of it. Educational plans are being., 
written by special teachers, regular teachers, and t eeate at meetings 
aera school principals. as Nicholas Hobbs has remarked, it | ; - ° 
. 1 . "og < r . 
may be much .like. rediscovering i weaiees Fails, phe eke schools are re- f 
: / . t . *” ‘ 
~ discovéring sareuteas We ave asking teachers to write carefully developed : 


. 


aa plans for students; they are not dlwaye competent ‘to do that 


Gur all of it is ielan done in an unprecedented public way. ‘ ; 


\ 


tte field of. leatning iieabestttees whighswve are considering at 
Cs this conference, is in the middle of te gil. It has been the fost . * 
. . . : 
tig “rapidly cover ob te aspect of special education in recent years’. Recently, 


ene has dean more attention politivally than atk nthiee fields of special : 


’ 


a “education combined. witn th there pis: vaguenéss. or uncertainty ‘about who | 


a * 


$ : 
_ these Hy carntag disabled" youngsters are, or how we define this Category ‘ 


5] 


of exceptionality, the policy Makers are very concerned. It is the 


first category in view when concerns. arise about limiting’ or "capping" 


the funds for special edutation. "Ss “ rN, 


Learning disabilities also gets attention because it deals with ye 


a 


the most fundamental areas of the school curriculum - what Stoddard has 


' termed the "cultural imperatives," such as language and basic: mathematics. 


We carry a larger burden, pérhaps, than any other field of special edu- - 


o « 


cation in charting new ways, yet there are weaknesses in’ our situation 


~ 


which create much difficulty. 


Y 


Let me mention just a’ few,of the special areas of concern which 


és e 


come to mind in the field of learning disabilities. First, it may be 
: ae ee 


s 


| ‘ a -s 7 : a 1l 


observed that’almost all qf the problems of. psychometrics or measurement [ 
come to the front--{n this field - problems of expectations, discrepancies, 


a. profiles, Yeliability, norms,.and much mores “Most definitions of leutiten 


disabilities start out with a statement about educability, sauatig in- 


‘ 


volving IQ test So, there.are all’ of the sutticeittes and furdeaa of 


Pa 


that concept of educability. ‘And then we proceed to noting Gaith only 


half an eye, since we pay attention to discrepancies in only one direction) 


y a 


. whether or not the achievement of youngsters is up to what’ might Be ex- . 
\pected or predicted. I was reassured when Robert Thorndike said in his. 


book Under-and. Overachievement iat it is hard to decide when the psycho-~ , 
logist’ has overpredicted and when the child has underachieved. “Phere are 
grave difficulties in dealing with. discrepancy. scorts as part of a 
basic définitien. -_ | 

, 4 ; et Fee a OM 


The field of learning disabilities, more ;than any other, is at the 


o ¥ Ps : - . 
‘center of a majér, transformation in measurement systems away from what 


‘ ‘ 


Leona Tyler calls a vertical emphasis, in teh we snphandesee genie inde. 
of predications about how high one might expect a oo go, toward - 

a. more horizontal emphasts, in which tthe concern is abt fox screening and 

sorting, but rather, on trying to design programs for the maximum de- 


velopment of each individual. = ee é 


. 


} Second, there’are ptoblems of interdisciplinary communication. 1 
am thinking of a — prominent article in the field of learning otsa> 


peiteiee: which on one page defines learning disabilities in terms of 
& 
a neurological base and on che very next page indicates. how lacking we 


‘ ~ 
~ ’ 


‘are in knowledge about interactions between, neurology and education. . 


. 1 
* 


Until we pet to the point’ “of being apie to specify Sch “ie pemaceous 


3? ‘ " ~ ‘ bd oO, / 
the knawiedea base a this porn - empty We have not learned how 


to scan or communicate. well across disciplinary aie professional lines 


s i 
and this is mot a minor problem. The amount of money that eoee) tute 


: teaming av aaeenanta, oe ane case studtes, and multiple 


a 


refetrals is very great, yet the outcome Xs often very thin soup. 


“ ‘ ‘ ; - sf 
order to develop eee eter scap i inany communication, there 


s 


mus& be concegtration on one dominant oe at a time. # Agsest 


chit in the field. of lehrning.d disabilities, the doa tact focus should : 
. 4 e 
be on the teacher's problem, that is, pn teaching and learning. 


° > Q 


ue have not communicated very well to neighboring dsssitaes and pro- 


fessions about the-decision making or the instructional axobland of 
. ‘ ’ 3 ai * 
the teacher. .I think this) research institute faces a large challenge _ 


of helping to sort out the logic of judgments and decisions made in 


the classroom and to communicate some of that to other disciplines 


so that per can learn to communicate more dePectively with - ceachane. 
& jo# : “+ ; ° 
Third, there are big problems, distal as well as technical, with 


. . t + 
ey 


respect to when'we begin treatment.\ There is a tendency in the field’ 
1 i . 7 


of learning disabiliries to wait around until we have a full blown 
; a 


. casualty - a big discrepancy - before beginning specialized treatment. 


We simply must seek the resources and structures by which we can iden- 
tify children who need special attencion earlier, differentiate: programs 
earlier, and ease the rates at which children learn successfully 

in the sears of: the cultyral smperalives: Specialists must engage 


potential problons: and not just full blown casualties. We must reduce 


_ the rate at which cha eben experience years of failure in the schools. 


However, when we move to éarlier programs, we face agafn the problems 


e n % . 


: > 
° , . 
. 
s 
’ 
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of definition, identification, and. "labeling; we run head on into funding 
= avneumeqwares require the big discrepancies. We 


%, deal with eroups of thildren at risk and to meat problems of 
ciety aie on a statistical basis rather than op. the basis of ° 7 . 
labeded individual eyige AR we procéed in udies of children with 
*. P.x° learning, problems we must bring: sone of. the 


‘stream and help improve progtams there. Som 
- _ ~ * , re 


nowledge Back e9 the sais 

of- our ledrning disability oo 
spegiglists could-well be deployed into deiabcecs settings in addition 

to, their clinf@al settings. As the Chilean poet, Mistral, hag Ssidh 


: a 
"Many things can wait, the child cannot. Right now his hip bones are 
a 4 2 ‘ 
being formed, his blaod is being made, his senses aré being developed. 
To him we cannot say 'tomorrow,' his name is today." 
i OR : 
‘ ats 


‘Fourth, there is thaggeneral problem of student Classification. ~- , 


_ When the ‘Congress asked eg ycators to come up with the ‘definition of 
F a 


"learning desdbitiey" au time ago, they made public a very considerable 

* ’ embArrassment’- that we don't havea very good definition of the-concept 
‘ | of learning disability. The systems we use now to define Learning E 

dteabitietes on vie practical scene or to define —. or : i 

eedihane mentally retarded are hot all that pene hes Peak Meehl 

says, the categories we use to define human Behaving do not "carve 


’ : nature at its joints"; they are, to a very colsiderablesextent, political 


constructions. We ought to work at the general problem of human classi- 


oe 


ficatidys and seek for better delineation$, such as. on may be needed 


Bs) 


for instructional purposes, in che schools. 


a . Fi : . 
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I launched these few remarks by saying that in this year we have ‘ 
" ge 4 . ’ ae * 


put a great deal oh the educational plate concerning handicapped 


‘ students. .We “f° challenged as never before. Truiyfit is a time for 
: t, tio . - ¥ 
“> 


concentrated reftection as we hope will be achieved at this conference. 
, 7 
, ) ° : . 
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This.paper is divided into three major sections. I'll-describe 


a 


f se z : 
some of the basic considerations* underlyiag -our research and take a 


‘ : o 
— 


ss : . hs Loe 
*) look at why we are focusing on the wr eal we a! process, . 
3 e o ’ i ae “7 z > ; ss a4 
Py . m : : . e . - 9 ; ‘ 
on assessment and decision ee and ia eia how we -see those as 


. . r - 


a * 
- * 


.». being interrelated. Second, L' u describe sai of the issues that . 

have essentiajly troubled us. regarding the Ee acsegent of learning 

~ disabled children, and talk about those issues as underlying premises 
» 


. > i iA ’ 


for the kinds off research: that we plan to be engaged in.» Third, I'll 


‘describe the overall rationale or structure that is guiding some of ‘ 


our research activities and then describe, very briefly, those activities 


O 
9 


themselves. 


Assessment in the Context of Decision Making 


We are defining assessment, not in the traditional ae | of 


’ 
id ‘ U 
‘ . 


. "testing," but simply°as the process of collecting data for the purpose 


of making decigiong about pupils. In educational settings, there are 


wf 


essentially six kinds of decisions that are made using assessment data: 


referral decisions; screening decisions; classification, eat 


te, 


tion decisions: and program evaluation decisions. Now it is important > 
from the outset, I think to distinguish Between ttose. kinds. of deci- ‘sh, ’ { 


a : . . J 
sions because it will lead directly to some of the issues we face. /. 


\ ° i This presentation is based on another paper: Ysse@ldyke, J.E. & 
,, Thurlow, M.L. Psychoeducational Assessment and Decision Making: 


A Review. In J. E. Ysseldyke and F. Morrison (Eds.), Multiple 


. perspectiveg.on assessment of learning gisabled children, in press. 


rd 
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When we talk about,referral, we talk simply about the identification ° 


a 


‘of children fot whom a referring agent believes there ia sufficient dif- 


% ae wane bi 


a F — 
s . . ficulty that some specialist ought to take a look at the child. ‘Screengng, 


« 


on the other tind 5 consists.of the administratton of tests, ‘usually group 
“tests, to scons ‘of childreri for the purpose of identifying See une: -, : 
Lat differ sufficiently from "normal," whatever oe nay Geeehak furttter as- 
re 7 sessment is believed ide cir ata “A third, aaa very, different, reason 
Le cuaaeias in the assessment Of children is for aie: purpdes’ of making 


* 


classification, placement, eligibility, or identification decisions, In , 


this case,. our questions are Muon One,. identifying the extent 
ie to which the child is handicapped; second, -specifying ‘the nature of the 
aes ae 
handicap; and third, identifying the least restrictive environment,for the 


a 1 So . . i J 

child. A fourth, related,purpose, but again a very different reason for 

da in assessment, ts for the purpose of planning either instructional . 
’ ry 4 


». 


.orfother kinds of interventions with children. Here, the questions we're 


+ 


Pi - J ° . - . 
| iia as assessment are twofold: We're ‘trying to decide what to 


e - ., > - : - 
peach, which is a content question, and we're trying to decide how to teach, 


» which is a question of the kinds of strategies, methodsy techniques, etc.,: ‘ 
t * : 
that will be effective with the child. . I should mention that there are 


subsets of intervention planning;; in some cases we'te concerned with imple-- 


‘mentation, how to “get an tear Pakelonal program going, and in other cases 
we're concerned with adjustment, what’ to do in the process of intervention - a 
4 " ¢ , “N : 

ara to modify or change things in order to move cheNgyete ahong. A fifth |. 


and very. different eason for assessing children is for the purpose of 


evaluating the extent to which they are making progress in their educational 


‘ 


programs. Parents, teachers, and children themselves have a very great need 


Qo. 
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to:know the extent to which progress is indeed being made. And», 


e 
a 


° finally, assessment data are collected for the purpose of program or a 
: . 7 7 u 2 . t e 


‘ ". " Yation, when we are attempting to evaluate the effectiveness of a parti-- 


cular. instructional program. This activity is best evidenced ‘in the kinds of: 
t 7 ay 


1, evaluations we've seem of Head Start Sroprans and specific curricular 4/ 
interventions. The whole precess of assessment isa deetslon-akiae ne 
es ‘opss' and one in which our assessment. strategies: and techniques ought to 
: loo be dictated, by tie 1ehnds of decision eects to make’. 
‘ a . : ; 


~ 


: aS 1, 7 . AY 
The Use of Assessment’ Data to Make Intervention Decisions 


Typically,:- traditionally, and too often currently, assessment and ° 


? 


— 
intervention are viéwed as mutually ‘exclusive activities. Educational - 
personnel speak of- assessment on the one hand, and intervention on the 


other, without viewing the two as integral parts of one dynafic process. 


The effectiveness of any specific treatment or intervention is the 


+ 


Nanction of a complex interaction of at least five identifiable factors: -., 


1. The characteristics df the child, 


4 


The characteristics of the teacher, 


a ‘ 


wh trp 


The nature of the treatmtent or {intervention employed, 


' « 


4. The setting in which intervention fs implemented, 


5. The kind of behavior change we are trying to bring about. 


4 


Child characteristics interact with teacher characteristics, which in - . 


turn interact with the nature of the‘intervention used, with setting 
\ - , 


a factors, and with the kind of behavior change we are attempting to bring 


tf» -' 


“oof about to affect the effectiveness of intervention efforts. We typically 
oversimplify a very complex process. 


In spite of the complexity of factors affecting intervention ef fective- 


ness, educational personnel are charged with the task of deciding which of 


a 


- 18 “7 


several alternative interventions to employ with a specific youngster. 
“ , "e , i ~b : - 
i , gust decteiend should'.be data-based decisions. © —_ : 


bt 


BS a . . 
Schools routinely collect many different kinds of data about children: 


scores on norm-refevenced tests, data from é¢riterion-referénced measures, 
ty € 7 

2 . » ‘ _ 5 \ 
observational data, interview data, medicaY information,:developmental ms 


s 


history data, social history information, and information -regarding 


*, , adaptive. hehavior. These ‘data are used- to make decisions, and the deci- 
= ' . mY a : : 
sions themselves are idee ventions: . “< 


Figure 1 illustrates the use of data to make different intervention 


decisions. The large box at the top of the figure illustrates 

several different kinds of information or data that are either available 

or may be collected for the purpose of decision making. Certain data are‘ 

used for the euseose of making screening decisions, other sets of aeea are 

° used to make iaktaane decisions, while still other data are uSed to make 
intervention aeeialiose: As is illustrated in the figure, the same data 
may be used in esing ve than one kind of deéision; more pre 


though, the data used do not entirely overlap. Different kinds of data 
: ' i . ® 
x ¢ - . 
are ee for the purpose of making different kinds of decisions. +: Assessment 


4 ; ¢ 
and intervention are not static, they are dynamic parts of the assessment- 


4 
> 


} intervention process. We believe we can best impact intervention by 
conducting research on the ways in which assessment data are used to 


make intervention decisions. | 


Insert Figure 1 abeut here 


Definitional Debate : 


' 


P There is, and has been, little consensus among state departments in 


the ways in.which they define learning disabilities. . This fact is illus- 


Qo_ : : “ 8 
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Perr peas ‘ : ay) 
’ 


t % ° 


trated quite adequately by data summarized by Mercer, Forgnione, and 


f 


Wolking (1975) and presented in’Table 1. Considerable variance in the use 


of the Federal definition, in the en of’ ‘intellectual level, 
ty 

in the, inclusion of process deficits, ‘and fi the inclusion/ea§Lusion 

‘of emotional, problems, is yreadily apparent. - 7 og 


‘4 
w 


¢ 
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to Insert Table 1 about here 


~ 


i: don't want us to spend the next two dave dees me alternative ‘de- 
4 


Zeon of ieagnine dleabilities. (that 'w0uld be counter= productive? “fue 


merous “investigators before us have tried to identify c of learning 


disabilities, and supposedly identified causes have rum, 


4 


'the gamut from spi- 
ne 
nal ‘injury, devisees imbalances, and neurological dysfunction to inap-, | 


ore 


propriate nurturance and, instruction. Furthermore, bajcattaatoie: have been 
* ¢ Fa 7 
unable systematically and consistently to differentiate learning disabled 


children from either "normals" or other kinds of hand{apped children. 
: , MpY : 
Clearly, major problems are evidenced in deciding thg population about whom 


‘we are talking: ~* | ‘t 4 


The Use of Tests” for Purposes Other than Those For fans they were Designed 


1 indicated earlier a assessment data are ded! for making many 


‘different kinds of peychceducational decisions. A: ha or, problem is the 


ue 


failure on,the part of, flagno dete personnel to dittérentiate thete assess-— 
ment -strategies, devices, and- techniques in PigHe log the kind of decision 


’ to be made. What I m referring to heré’is eave’ ‘eke "WIS C/Wide = 


‘ 
for every child, no matter what decision we're trying to make.' This is 
probably best illustrated by the use of profile’ analyses in efforts to 

, . b 
‘plan instructional interventions for children. some of the best tests 


that are available are intellectual devices. “Thay were originally: designed 


! 


0 ee : - | 
for the purposebof helping’ us = classification and placement decisions, 
They. wii, ana dean axtremely sefeative job of helping” us make classifica- 
tion and prASemene decteten But, we WEERESS today ‘individuals etigaging 


ingelaborate profile atiatesee date devices Like the Wechsler Intel tigence 


, aN ° 
Scale for. Childreft insefforts to pian: taseunetional: Programs If you go 
ae 


v ‘ v 


and pick, uw a psychological report, it'll “read, “Johnny ' 8 poorest perform , 
ance e was deacnaceabed on a task requiring him to answWar specific factual 


" questions, while his best performance was demonstrated on a task requiring 
; zi - a 


ts, 


hi to repeat sequences: of orally presented digits. Johnny outa RECEDES 5 


eon a prcaeall in which the teacher ‘would Nave him answer many specific fac- 


yy 


tual questions." Yet, there is no enpirieal support for that practice. 


Technical Adequacy . i : ‘ 


‘ Technically inadeqnate notm-referencedstests, are too often used to pavner 
a f . 
data »fon -the pur tte of maine amportant decisions. Three factors are in- 


volved. The irst is Leandardisation: When we assess individuals, we as- 
; » . 
sume that tHe individuals we assess are like those in the normative rot 


that the task has een. “Seaudaveived on, children ane are like the child we're‘ 


O 4 


s 


assessing. We assume that Efe indtvidual’ has Had a comparable set at -exper- 
feneesS and opportunities ‘to learn. A good numbér of the devices that we use / 


on a daily basis to make decisiong about-.children simply don't provide us 
° e j . 4 * ‘ 

with a set of.norms.. Table ,2 lists tests with inadequately constructed or 
> ‘ 7 7 ‘ . : 

described norms. a ea 


a 
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The second issue is one of reliability. Mast of us, somewhere in our 


careers, took a measurement course, and we learned that assessment devices 


should be reliable. The commonly accepted atdhdard is that a reliability of 


. ; re he 
“ - Pe ; Poy 


vere a ae ee ee eo Ut ek, ee, a eT 


-90 ought to be deweweisaben peters devices are used to mgke.important 


© ‘N 


deciatons about individuals. apie the devices that we use to make de- ] 


4 


cisions about pupils lack the necessary reliability for effective use. 
. . ty 


Table -} lists reliabtlities of commonly used tests and indicates that for 


the most part individual ‘ group iftelligence and achievement measures 
‘ 7 , t ‘ . 


do have reliabilinige that are somewhere in the -90's. Reliabilities are r 


a 


all reported in ‘Tanges because they are taken dErecthy from the test man- 
uals and reflect differences across age. Reliabilities of measures of Spe 


cific procasses or abilities in no instance exceed .90. Sufh degiece stoutd 


4 
e ° 


not pe used to make important decisions. The third issue is validity. In 


measurement courses we all learned a little ditty that says "Reliability is. 


a necessary: but not sufficient condition for validity.’ Devices that have ~ 

, f. ‘ 
inadequate reliabilities cannot ,be considered valid. Table 4 is | a ‘list of 

Pa i . 
tests with questionable validity. ( 


ee ee a cs ee eee cr ee es ee ee re ee ee ee 


Insert Tables 3_and 4 about here 
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We've got some major problems when we start looking at the technical 


adequacy of the devices we use to gather data for decision making. I used 
, , “e : P : ri . 


to believe the notion that it's better to use a non-reliable test than to 
use no test at all. Yet, such tests are indeed dangerous because they af~ 


ford the illusion that they are providing reliable, valid information. 
Bio: , 
\ 5 ee “ 


— | Using Deficit Scores + : . . an , 
The. next issue is one of the use es: scores to identify the “+ "s 


learning disabled. The reliability of a deficit score is nearly always 


lower than the reliability of either of the scores’ that Zo into it. -So0 
‘ wi . 
when we start using non-reliable devices, and then put those devices into 
ae 


“an equation and compute the deere of deficit that a child demonstrates, 


‘the reliability = are > dealing with is eteothreant’y ine foe cua have 
| 


4 ‘ : “| ra \ 
a e 7 : 9 : 7 . 
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been able to show that reliance on the use of deficit scores results \ 


ot a significant amount of mis-identification simply as a function of change. 


- Bias in Assessment és 


Be a if ade oe e « ° 
oe ‘ ‘ 


This nation has spent an enormous amount of time and effort in attempts 
© : to identify the reeb that ‘is fair Boeulise with members, df. specific racial , 
| ; “or cultural groups. . When ri in Pennsylvania, baaiuaic in the State 

1 : . Department were engaged “in the task:of coming up with a list of Beate that 

you could use with children who were members of specific racial groups. We 
a, darned from a history of similar efforts in psychology, pavéhclo= 


ve long debated the concept of "fairness," and have deVeloped many 


mathemat 1 models to compute the extent to which tests are fair. The 


one th / that is very clear to us at this point in time :is that there is 


very tle agreement among those who have talked about models of test fair- 


i , . -_ - ny 
ness, .and very little common consensus on the definition of non-discriminatory 


e 


* 
assessment. . 


oy : ab 


We could suddenly have the fair test, a test that was fair for all 
shiideea. or we would still have considerable bias in the assessment and 
ecitora process. _qiesearch has demonstrated that if you take Saves: 
ionical reports and put the picture of ‘an attractive third grade child on 


‘the top of one Devehere sce report, and put a picture of an unattractive 

; <5 | 

» 

third grade child on the other report, “teachers and sntervepagon agents make 


gia sdifiérent kinds of recommendations and different, kinds of “prognoses for 
those children. ‘People i deeds toncnei(ag ee ruarrors do imdeed discriminate 


‘or bias on the basis. of things like Bex,” “ptiysical attractiveness, parental 


power within the system, SES, and so on. . , ' 


Placement Team Decigion* Making 


z , o4 [ae ON ys. “ 


7 A set of issues is {directly relevant to the placethent team decision- : 
; : P 


a, . * 
making process.’ The‘point-in time.where Nesssauens comes together. is 


. < * «4 Sateen f oe ‘ \ 
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v ~— 


when a group of’ individuals sits down for the purpose of presenting data . 


e 


. : and tries,to arrive at recommendations regarding the placement of the in- 


dividual and ‘the kind of instructional intervention that is necessary: for 


Lg x 


that pupil. When we:look at research relevant to the placement team deci- 


sion-making process, there hasn't! been a lot. There has been research on | 
ry . 7 - 
clinical judgment. Research on clinical judgment has typically employed 


a human information processing design, and has largely been inconclusive. 


The body of research that does exist indicates that ndividuals- typically 


F ‘ & 
make very HEPeEr age one We face the task now of trying to. see 


. whether individualsewho make unreliable decisions will suddenly achieve 
2 
réliable and indeed valid decisions as-a group. 


A second body of research is research that has useda questionnaire- 


, interview methodology. Yoshida, Kaufmann, Fenton, and their colleagues at 
a 


. 


’ the Bureau of Education for the Handicapped, conducted a series of investic 


‘ 


gations in Connecticut where they asked individuals who had participated on 


° 


placement teams the @xtent to which they kealy participated and the extent . 


‘ : ieney 


to which they were satisfied with the process. Interestingly anata. that 


set of investigations showed that teachers were less satisfied with the 
\ a . . oe ° 


~ _\ 
whole process when they attended decision-making meetings than when they 


had been left out of them. In the first place, teachers felt that they 
* = e 


didn't get an opportunity to-participate. But the important thing, the 


only point J want to make, is that the only data we have from those investi- 


yy 
{ 


gations are date that give us useful information on perceived participation 


(how much you thank you participated, how you felt about it), and the data 
¢ “38 


are not direct data on the process itself. 


A ‘third line of investigation relevant to the placement team decision- 
gl 
making process ‘has been a series of eGaputer siiulat {ion Seudien, some of 
A , - - 4 
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the reading research oF Schulman and Vinsonhaler. and’their aia 
, P , E v 
at Michigan State, Janet Lerner's program that is uged to train indi- 


viduals to: come Into consensus with expert opinion in diagnostic situa- 
4 . : , . 


‘ tions, and somé of the research of Alggzzine. ‘In such studies, by means 


of the computer, people have been provided with data and then asked to, 


make decisions about indiyiduals. There are some real questions about 


the extent to which you can indeed simulace the dectatonsmaking process 
by means of deatpaken, One of the interesting sidelights, I think, to 
He research that. we plan to-do, is that we’ plan to study the same indi- 
viduals by means of simulatton, dhesivatien, and quest iognaire/intervieu 
procedures..’ We ay be able to get a made on how much simulation really 


approaches reality in terms of telling us anything about the decision- 
v oe ‘ 
ety ry 


4 


making process itself. P 


The fourth line of iNvestigation consists of use of systematic, 


naturalistic observation of the placement team decision-making process. 
” . 


* That methodology, I think, is terribly attractive, but if's been relied 


on very little. We found only one investigation. where people had gone 


} 


my 


in and looked at the placement team decision-making process. Patton 


(1976) conducted a study in which he looked at what happened, and he 


% 


indicated that the placement team spent about five minutes: per child in 


making a decision, and that the decision-the school psychologist came in 


with was the decision that th@ team ended up making. 


ie 


3 


a 


The‘ last relevant b of research has to do with decision-making 


¢ 


models. We did survey the decisionrmaking models; we found difficulty 
- applying them to. the kinds of questions that we're asking. There are 
4 ‘industrial .decision-making models, the group process models, where we . 


look at how groups arrive‘ at decisions, and so on. There is quite’ an ee oe 


> 
4 


extensive body of research on how teachers make decisions in the instruc~ 
tional process. Yet, we're having trouble finding models that fit our 


research. 


g . - " 6 


Rationale for Research on Assessment and Decision Making * 
% The foregoing review indicates clearly that there are many major 


questions regarding assessment and decision making, questions that are oe 
Nov t : . 


Pa 


obviously worthy of investigation. We have(chosen to engage in several 7 


oy 
v 


‘lines of inquiry as a basic beginning for ou longer-term research efforts. 


It ig our belief that the questions in this area can best be 
addressed by conducting several interrelated investigations. An. over- 


a 


riding perspective guides our reséarch on the placement team decision- 


making process: We believe that we must conduct our research in the e i 
naturalistic environment, endeavoring to achieve what Bronfenbrenner ” 
4 : F 
(1976) calls ecological and phenomenological validity. 
° {? 


Research on decision making to date has been largely contrived . 


i , 
Da research. I don't know howmany times in the last year I've: received 


bogus case studies in the mail where people say, "Here are some data 


. Z 
on, a child; we'd like to have you make a decision about the child." , ’ 


The approach gets used so much now that people are becoming very sensi- ” 
~~" ; yd 


. tive to ft. Individuals have looked at the influence of parents’ 
wavibal stave; racé, sex, socioeconomic status, and so on,,on the 


decisions that, are made, but it's a contrived kind of situation. The 


1 — : r : { 


. 


‘data to support those beliefs, “ideas, and theories, Such data, we 


‘disabled in order to provide ay with the services of a teacher that 


26 E ee ee - \ 
case study, and in-dab atmilakious by means of computers, have been . 


2g: -_ ° . 
used to try to study real-life phenomena, The generalizability in 


¥ 
those cases is presumed but never demonstrated. As & result, it's 
. a 


probably safe to say that we really know very little, if anything, . 


- \ Bi as ee 
about what‘really happens in decision making, We have some beliefs, 


T 
~~ 


some general ideas, and some pet theories. We've no ‘ecologically valid 


believe, will, be obtained by studying the decisioh-making process 


naturalistically in the environment in which it occurs, and by enédeavor- 


ing to document and understand the many complex factors affecting the 
ee wn, ’ we 
process. or) h a + te x 
. | “ i 
- But We also can't *simply go out and obsérve behavtors and count 
— ; b 
beh wlors that occur during the decision-making process; we have to * 


strive /for a second principle, what Bronfenbrenner (1976) calTs phenomeno~ 


logical validity. He defines that by saying an ecological experiment - 


cannot be solely behavioristic; provision must he made for assessing . 


each participant's def init for of the situation, how he or she perceives 


the setting, and its various elements. In naturalistic settings, far 


« 


more apparently determines. the behavior of decision makers than mere 


‘ ° 


reliance on objective data. Decisions are ‘made as a function of the 
ee ceainee ieee the Si 1aeei, » be sure, but fei tone are also ' 
made as a funetion ét eketng teks beliefs, attitudes, motives, maybe 
even ‘the reinforcement history of the decision makers. Decision makers 


+ 


have indicated that they've very often labeled youngsters as learning . 


they believe is highly competent. spaitety, they state that they re- : 


fused to label youngsters as learning disabled when they believed the 


J 27 


¢ 


f services of a particular program or teacher were not in the best in- 
c i : 


térests of the child. Merely Dserving the behavior of persons in» 


decision making wil] not enable us to discover all of those factors ~ 
\ : . . 
we want to discover, it will be necessary to personally interview the 


_ * 
decision makers to uncover those ecologi¢ally relevant factors. Thus, 


» ’ . & . . 
— the research will emp)ay several methodologies, whefe we not only go 


out and look at what happens, but we also ask people what happens, 
4 ; : . baad Pi 


what they typically .do in the process, and so .forth. 


Proposed Research 


4 


ay 


am ‘ 
. i ’ 


: There are six lines of research in which we ‘are involved. The 


highly syNematic instructional program, and you put children who are 


identified as learning disabled into it, a lot of children improve, but 


« - ‘ aa 


there are always a group of children’ for whom success comes only very = 


* 


slowly. Some of our questions in thts research aréa are:* Can we, ee? 


on the basis of data that are available, at entry, differentiate children 


ll yee 


_who are successful from children who are not? How do you define success 


J in a wystematic instructional program--in terms of number of objectives 


€ Py ~ 


accomplished, in terms of return tp a regular classroom situation, or 


: . yy & » 
whit? 
f é 
é 3 : « ~% 
A second area of research is a series of computer simulation in- 


. 


vestigations. What we're concerned wi ete is looking at the assess- 


—_—_A 


ment process itself,as a function of the kinds of information presented 


« 


to the assessor. The computer affdrds us the opportunity to systema- 


b 4 


tically vary input data and to systematically vary the sex of the child, 
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‘making decisions, provide people with data, and ask them to go through 


28 °° . 
~, é "y bo 


P ‘ 
the physical attractiveness of the child, and Beene) and then look at 


< 


the dectpion-aaking process-~look at the tests that individuals decide. 


‘ 


4 


to. use, look at the extent to whitch they access’ technically adequate 7 


Aare on the test, and look at, indeed, the kinds of outcomes that are 
rae , : 


achieved as a function of that process. We will simulate decision- . 


4 


‘the decigion-making process itself. We also intend to look at that ‘ag 


« 


a functign of the knowledge base regarding assessment. \ 


A third research area is Teally a two-fold study. We are conducting 


comparative research on children who are failing academically and who are 


‘labeled learning disabled and on children who are failing academically and 


a 


who are not labeled learning disabled. How are those children different? | 


? 


Psychometrically and demographically, what are the differences fn those two, 


, wt ; 
groups of children? We are really asking to what extent do those ‘children 


demonstrate differences in their performances on task, to what extent do 


+’ 


those children demonstrate SES differences, racial differences, sex differ- 


‘ences, differences in physical-attractiveness, differences in the power 


‘ ferentiate the groups? : . 


that individual's parents have within the school system? How do we ddf- 
yw 


4 


The fourth line of research is a series. of questionnaire investi- 


gations where we ask decision makers. how. they: make dee tetons. One in- 


ies is designed to look at Ene eee ‘building principals decide 


A 


ow to put childrén in different Sideaen’ ‘in the first place. Do people 


“use rules to make those decietons, and if so, what are the rules that peo- 
ee Y - 7 7 


ple use? Secondly, we intend ta characterize the aan a process in the 


a 


. 


re 7 ‘4 C : 7 29 
‘ model demonstration centers, looking at the cir of assessment de~ 


’ 


vices and procedures that are, used to make decisions and the extent 


° 


to’ which those assessment procedures are differentiated as a function 


‘of decision making. | In the third line of Dei. we will 


‘ask special education personnel to characterize the decision- 
f ’ < ‘ 


v 


4 
4 ’ 


making process for us. t happens, all the way from referral to 


outcome? 


a 


The fifth line of sesearch will allow us to check out some of the 
we. , ‘ ° » 
a - 4 
’ tnformation we receive,from the’ other lines of research. It involves. 


oO 


ecologicai research on placement team'decision making. We intend to 
observe placement teams in the process.of making decisions and to look ) 


at the kinds of decisions that are made. We will look at the}amount a 
« % 


of time spent in presenting data, dnt one eatee and, also will con-* 
" t — 
“trast that with what pecpie say happens in the decision-making process. 


s 


ks _ We want tod look at the kinds of data that are Sueeodneed and try<to 


get a better handle on the process. 


The last research area involves the notion of bias following assess-— ' 
ment. We will ‘look at the influence of, having an LD sikling on later 
' al > 
; teacher behavior toward and is Aenea regarding an LD child, and 


we will .look at-the extant to which those . expectations may differ as 
a function of social and academic’ considerations.. ; « 
‘ ; i, f 


= : w ss : i 
. 4 
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_Number of States and Respective Percentages of Components Includéd in State Definitions’ 


\ 


> 


No. of 


Components Skates 
Definition . | * * : 
NACHC only ° 9 21.4% 
NACHG with variations = -.15 . 35.7% 
Different , - 16 38.1% 
None = 2 4.8%. ° 
Intelligence 
Average and above . 11° 26.2% 
, Above mental retardation, 8 9.1% 
Not stated 23. 54.8% 
Process © , : 
Process disorder 360) 85.7% 
Language disorder 35 83.3% 
Academic : 
Read ing a : 31° 73.8% 
Writing, — . * 31. 73.8% 
Spel. Ling 31 §=673,8% 
Arithmetic 7 31 73.8% 
a 
Exclusion-- primary 
Visual impairment 26 =661, 9% 
Auditory impairment 26 =61.9% 
Motor impairment 23 -54,8% 
Mental retardation 2 50.0% 
Emotional disturbance 25\ ° 59.52% 
54.8% 


Environmefital disadvantaged 23 


br a 
From Mercer, C., Forgnione, Ci, & 

the United States. 
> . 


ERIC 


Wolkikg 


Percent 


UNS | OO 


Components ; 
Exclusion - primary & secondary 

Visual impairment : 3 

Auditory impairment 3 

Motor impairment ~ » 2 

' Mental retardation - 4 

Emotional disturbance 1 

1 


. Environmental disadvantaged 


Neuralogical impairment 


_* Included - 4 
“* Not included 0 
Possible e 4 26 
Not stated a 12 


Affective , 
Includes emotionally 


disturbed. ~- 4 
Includes socially mal- 

adjusted 6 

Miscellaneous : 

Attention deficits oe 

Motor deficits: . 7 

‘ Thinking deficits 30 

Discrepancy component 12 


Special Education required 14 


. Intraindividual differences 4° 


Prevalence 2 
Chronological Age © : 4 


: | 


g 


No. of 


» oe, 


Percent 


9.5% 


14.3% 


11 e 9% —_ 


16.7% 


71.42, 


28.6% 
33.3% 
9.5% 


"4.8% 
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» W. D. Definitions of learning disabilities used in- 
Journal of Learn ng Disabilities, 1976, 9, 376- 386. 
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Table 2 


” » 


Tests with Norms That Are Inadequately Constructed or, Described 


en ee 
a 
Arthur Adaptation of the Leiter International Performance Scale (13)* 
Bender Visual Motor Gestalt Test (15) | . & 
California Achievement Test (9) 
Culture Fair Intelligence Tests (14) 
Cognitive..Abilities (14) b 
_ Developmental Test of Visual-Motor Integration (15) . — ws 
- Developmental Test. of Visual Perception (15) ss _ \ 
Diagnostic Reading Scales (10) : 7 b er a 
Durrell Analysis of Reading Difficulty (40) . 
Full-Range Picture Vocabulary Test (13)> 
4 Gates-MacGinitie Reading Tests (9) a 
i -Gates-McKillop Reading Diagnostic Tests (10) 
, Gilmore bral Reading Test (10) 
.  Goddenough Harris Drawing Test (14) ; 

Gray Oral Reading Test (10). . 

: Henmon-Nelson Tests of Mental Ability (14) wy 
- Illinois Test of Psycholinguistic Abilities (17) 

Memory’ for Designs Test ‘ (15) , : ; 
Metropolitan Achievement Test (9) ; . ' 
Peabody Ficture Vocabulary Test (13) 2 
Primary Mental Abilities Test (14) ; 
Purdue Perceptual-Motor Survey (15) : a 
Quick Test 413) : (43 . ‘i 4 
Silent Reading Diagnostic Tests (10) 4 : ‘ 
S$losson intelligence Scale (13) .° 
3 ; Stanford-Binet \Intelligence Scale (13) 

' Wide Range Achievement Test (9) fe 2 
Tg \ 


“Numbers in parentheses refer to the chapter in which the test is ae nr" 


. 


. a, . described. §, 7 ; ; 
a ae , brnese. ‘tests rociage norms in their manuals but include no data = ; 
about the grovp on whom the test was standardized. 


et Ry I 
24) 


33 , 
; ; Table 3 4 i 
‘* Reliabilities of Frequently ‘Used Tests ae “4 . ae’ 
: . ; 
! Measure > : ty a -. Reliability 
California Achievement -Jest (Subtest Relfabilities)’  .76 - .977: 
° Iowa Test of Bagic Skills (1974 pdicton) None - te 
Metropolitan Achievement Test , 84 - 6965 
‘Stanford Achievement.Test (1973 eg tiog) - 65-4070 
.Gateg-MacGinitie Reading Test . | 88 = 965 
Peabody Individual Achievement Test . “ 42 - .94 
Wide Range Achievement Test. as 
Gray Oral Reading Test . eg OT = 2985 ‘ . 
Gilmore Oral Reading Test ‘ : -53 - .94 See x 
Gates-McKillop Reading, Diagnostic Test ' None. | : : 
Durrell Analysis of Reading Difficulty. . None : _ 
Stanford Diagnostic Reading Test (1976 edition) 75 - 94° Oa 4 ‘a 
Silent Reading Diagnostic’ Test 85497". 2 © 
'. Diagnostic Reading ‘Scales = ne «87 - .96" : 
Woodcock Reading Mastery Tests 7 679 = .995: * 
Key Math i | - 39.908 2 YN, 
Stanford Diagnostic Mathematics Test 7 ge 2 .84.- .979" 
- Stanford-Binet Intelligence Scale - None 
; . Wechsler Intelligence Scale for- Children--Revised, a 
4 Verbal. * F é ; 91 ~ 96 
/ Performance; ~~ ~- &, a - ' 89 - .91°, 
Full Scade ee so > 95 - 96° 
Subtests ° : ' .62 - .92° 
Wechsler’ Adult Tateliteance Scale * _ ‘e ; a 
Verbal" - 2 496° = 
’ Perfotmance-. " * 194°- 94° . ' 
Full Scale : ’ 97° a 
Subtests | : ; a 60-960 7 . 
Wechsler Preschdol’ and Primary ae = & de ow . 
Verbal. : 93 5950, 
Performance . , 691 = 495 a 
Full Scale: 96% 970 
Subtests . wee : 7 %.62 = 91°) .e- 
McCarthy Scales of Children’ 3 Abilities ae ee ee * 
‘ Verbal Pa wa N86 = 920 . # oy 
Perceptual-Per formancéty rae o | , «73 = 590 . 4 
_ General Cognitive ee A RS .090 + 94° , 
Quantitative : a a 77 - 86% 
Memory “ _ fo. .72,- .830 


‘Motor . : a .60 - .84° 


. 


o 


4 
3 ei J \» : r ; 
; 34 ae 2 oe . -- . 
woes — “ . ¢ ‘ . . 
ba 4 
"> Tableg3 (continued) 
; = 4 
Measure _, Reliability + 
t . 4 : 
Full, Range Picture Vocabulary TBSR : . None di 
Quick Test ’ _ .60 - +96, | 
Peabody oe Vocabulary Test , « .67 - 84° ’ 

- Nebraska Test of Learning Aptitude y, : .92 - 95° 
Blind Leatning Aptitude Test /° : 939 
Arthur Adaptation of the Leiter International ; 

Performance Scale a None 
Pictorial Temt of Intelligence = . .87 - .937 . 
Columbia Mental Maturity Scale | . 85 - 91% 
Culture Fajr Intelligence Scale , 7 bo! 

Scale'1 Total : . 80° - 

Scale 2 Total : - = TL = 828 

Scale 3 Total — S “ 51 - .68 
Coghitive Abilities Test (Total) a ' .91 - 95, 
Goodenough-Harris Drawing Test y * ; -60 - .70 
_Henmon-Nelson Intelligence Test (Total) _ 84 - 977 

. "Kuhl mann~Andergon Intelligence Tests (Total) .93 - 95° 
Otis-Lennon Mental Ability Test (Total) .88 - 96, 
Primary Mental Abilities Test (Total) -86 = .95_ 

- Short Fornt Test of -Academic Aptitude (Total) -90 - 96;: 
Bender Vigual Motor Gestalt Test (1975 manual) . -50 - -90,. 
Developmental Test of Visual Perception (Subtests) "  ,29 - 4/0 A 

Developmental Test of Visual Perception (Total) ‘69 b 
Memory for Designs Test ‘ : -72 - .90. 

'. Purdue Perceptual-Motor Survey ~~ None 
Goldman-Fristoe Test of” Articulation fe Inte-tater Only 

“Wepman Auditory Discrimination Test None ° 

Northwestern Syntax Screening Testi. ee None 
. “T1Linois Test of Peycholinguistic Abilities b 

a ' Subtests . it ' 12 - “90, 

‘Total’. . | 666 = .91, 

. ; — ‘* 
; a = 
“Internal Gonsistency 
“brest-Retest’, 7 : ox 
“split-half- ’ . 
“altérnate form $s 


4 


. a. a’ ‘+ . as —_ ’ - 
Source: Data ‘are from test manuals and were compiled from data reported 


, ™ earlier oY Ysseldyke and Salvia (1974) and Salvia and Ysseldyke 
(1978). : . 7 


« 
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' Metropolitan Achievement Test 
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- Table 4 Pe _ _ 


i 


Tests Having Questionable Validity* : = 


ad « 


7” : sng 
Bender Visual Motor Gestalt Test 


‘California Achievement Testb 


Developmental Test of Visual-Motor Integration 


‘Developmental Test of. Visual Perception - » 


Durrell Analysis. of Reading Difficulty” 
Full-Range Picture Vocabulary Testb 
Gates-MacGinitig Reading Tests . a 


‘Gates-McKillop Reading Diagnostic Testsb 
Gilmore Oral Reading Test 


v 


Gray Oral Reading Test 
Henmon-Nelson Tests'of Mental Ability . 
Illinois Test’ of*Psycholinguistic Abilities f 


Purdue Perceptual-Motor Survey _ | - 7 8 
Stanford-Binet Intelligence Sdale> . 
Wide Range Achievement Test 

> 


ee from Salvia and Ysseldyke (1978). 


No validity data are included in the manuals for these tests. 
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_ Use af Data to Mafe Different Intervention Decisions. — | i 


Psychoeducational Assessment and Decision Making: A Reaction 


. 7 7 
Sa iaes K. Keogh 


University of California,.Los Angeles . / 


Research within this section of the Minnesota LD Institute is focused. 


on two major lines of work. One involves’ inquiries directed dt. the charac 
—_ ; 
teristics of learning disabled children; another deals with the characteristics 


. 


of decision makerg and the dectsionsaune Wenecee: I would like to emphasize 
mye ; 

my enthusiasm for these research directions, and to underscore the importance. 

of systematic study of assessment procedures and of the people who make deci- 


sions about children. I was particularly impressed with the utilization of 


-@¢ wo 


tal 
a variety of data sources. Few people study principals, teachers, School 
psychologists, and children. ~- Most study only one of these. groups. I commend 


you for broadening your base of information and for utilizing different kinds 
‘ \ . “4 : . 


of information. 


As I see it, the fundamental question being addressed in this series: 


t 


of studies i's "What is the link between assessment data and educational de- 


cisions about LD children?" Answers to this question require knowledge about 
<a : ‘ 
children and knowledge about decision makers. Thus, the research plans 


appear to be on target. What has not been made explicit, however, is the 
. : a 


overall conceptualization of the linkage between the two sources of informa- 


ry ‘ 


“tion. At the conclusion of this research program you will Idgu a considerable 


' to this point that Tf wish to address several brief remarks. 


,@ ’ 


‘amount about assessment, about the’ technical adequacy of instrumentation, 


and so forth. You will also know something about, decision-making procedures. 


” 


at lgast, toy 


But will the data allow us to understand, and inferentialls 


ty 


Admprove decision tnaking vis a vis children with learning problems? [t is 
x * . 


e 


At leagt three well a ae approaches “to the study of 
7 


38 / 


‘ ara 


decision processes are applicable to decision making within an educational 


° 


to, getting. I refer specifically to the work of Richard siavenal, on teacher's 
‘ ¢ 


M 
r) r F 


“decision making, to the work of Bernard Weiner on attribution, and to 


the social power research of Bertram Raven. Although the latter two theo- 


‘ n 7 
* retical approaches were developed within somewhat different contexts, all 
, three seem useful for clarifying and understanding educational decision 


making. 
Shavelson views teaching as a dechanci-nae ine process. Based,gn 
i "work of other theorists, he has identified some héuristics which teachers 
use Be synthesize the complex informetion and the variety of data available 


when making {nstructional decisions about children. Shavelson's heuristics 
are appropriate in the decision-making context in whch your work.is based. 


” As example, Shavelson talks about a "representative" heuristic, an "avail- 
' | od 


\ ability" heuristic, and an "anchoring" heuristic.: To illustrate the*poten- : 
tial use af this ‘approach in your work, the first heuristic implies that 


when given. new information, we end to accept and credit that which is con- 


’ sistent with our view of the top 6s Specifically, if a youngster’ matches 


= oy ‘ 
our view or is "representative" of uf view of what mentally retarded chil- 
\ @ 3 


dren ‘look like," we are apt to accept\information consistent with that 


diagnosis, even when the information ‘is ufreliable. Such am heuristic 


may help explain why some bits of unreliaple information might be accepted 


« 


ess when otherg are discarded. — a, 


V 


and utilizdd in the decision-making pro 


‘ ) 

‘on by decision committees in special 

ra, 
'y) 


we 


Analysis of the utilisation of informa 
silipaiton- tight, be facilitated and clafified aaearal this heuristic. - 
uy apseemtiely: understanding of this heu jebie also ties the child's char- 
| acteristics to the decision. The Stier bao heuristics touch on somewhat 
: different yet related aspects of selection and ut{lization of information 
QO | é 
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in decision a aie but the overall point fo be emphasized is that 


there are a fumber of ways information is utilized in decision nfiing, 
and these processes need specification and study. . / 


‘ I would like fo see you generate more information about “the nature of 


the decision-making process in terms of the persons who make ae! Sia 
{ - s 


t is 
You have mentioned,that we use’ data to make decisions, and that the de- , 


is a 


. - cisions are influenced by the nature qf the data collected, I would argue “# 


ae : 
* further that the data you choose, and how you interpret them ate less a 


; function of the®technical adequaey of the instrument than of the "set" of 
r ‘ ‘ : 7 4% 


. the person who is doing the selection and making the decision. This aspect | 
i oN ‘ 7 
) of the decision-maki§g process is deserving of attention and of systematic 


~ 


investigation. 7 . 4 
‘ r , wo ; 


- In-this regard, the work of Bernard* Weiner on attribution also seems 
ro ‘ 


—_ directly related to what you aré doing. Weiner has proposed that in syn- 


: “eo , 
thesizing information about individuals, we. tend to make attributions a 
* . ‘ a ‘ 
, vw 
. about the causes of their condition. When a child has been referred ‘as aa 
ae ie "having some sdért of failure acl we. immediately begin-to seek reasons 


v 


for that failuré. Weiner has proposed that in most achievement sf{tuations, 
we make attributions to stable or unstable causes and to internal or ex~ 
, + ” ) 


ternal ones. We may say that a child is not doing well because he or she is not 


very smart, thus making ay attribution to abjlity -4 an internal, stable 


° ~ cauge. If.one views a child's failure in school as being due cosiade ; 
/ ae OE ability, then the speiene thing to change is the naare of the program, ¢ : 
es the curriculum, or the placement, i.e. , put the child fe a special class, 
. ~ r somehow to change the sducattonell environment to be consistent with the — 
) “child's presumed sbiidey level. On ci eee hands {f we make an attribution 
x ae motivation or to some ieee atcite Wind of characteristic, we are apt to. 
re) o . 7 4 ae. ’ 
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she'd only try,” or, "He really worked hard on that day." If the child 


he or Hee 4 , Be ST a a _ RN Nee ye ey eG 
: ’ 
by 
. : : 
P ‘ 
. 


40. is ff J 
ar renPe to change the child, i.e., to get the child to "try harder." ahe 
patat is ai cacndes make decisions about what ‘they are going to do in 
terms of their attributions abel causes of a child's disturbance or | 
problems. I suspect that decision-making teams make the same kinds of 


attributions in terms of “stabiesuustable; internal- xterng] characteristi¢s 
‘, : . \ 


as proposed ih Weiner's model. Thus it seems thé attribution approach — 
might provide gome power in identifying the dynamics that go on within , 


the decision-making team. | . - * (" 


> 


We have besun to explore this in some of our week at UCLA, and I can | 


a “one ‘¢ 


attest to the power ofthis model. Ina sendy by Lavelle, we, provided 


a 


_ achievement performance information. about their own youngsters to parents 


of mentally retarded, learning disabled, and norma children. When we 
asked the parents of normal children why their children were successful 
or unsuccessful on a particular task, the answers were as you might “expect: 


When the child was successful, the parents said, "He does well on that kind 


‘of assignment." Tf the child was unsuccessful, the parent tended ‘to write 


a“? 


that off and said, "Well, he had a bad day," or; "He didn't under gtand 


‘that particular question." When we asked the paregits oe Hearing disab fed 
; 
children the same quesfions, they made notivatiounl epeeicedona in almost 


all cases. If a child did he they said, "He's really a smart kid, if 


py 


failed, the parents said, "That's one of his problems. We just can't get 


him to buckle down and. really stay with something." The parents of the 


a 
mentally vavarded eect however, mad¢ what is really a very sad kind 


of attribution. When the child was wieneeesstel the Serents said. "He 


Bo 


{s retarded and he never does very well | on schol, tasks." When the child 


» é 
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was successful, the parents were apt to write that off and.say, "He must 


have been lucky," or, "It was a very easy kind of task." Understanding 


the nature of the attributions of decision makers is ea tapateane facet 
‘ ; 
of your research. The attribution model seems particularly well suited to 


1 4 


fy . ° i! 
: . decision making. Pas social power model may provide some understanding of 


‘ 
x 


' the interpersonal dynamics within dectsion-making teams. Raven and his 
ras colleagues, working within the social psychology field, have studied group 


processes in terms of social power. In their view, individuals influence 


a 


N 


other people on-the basis of a number of possible dimensions --' coercive 


” - 
re 


N ‘ 
power, legitimate pywer, and referent -power, to mention only three in the 


Raven@fodel. "Social power" Wimensions may provide insights into why 
some parents’ have more influence on school ‘decisions than do others, or 
why cegeata members of the decision team are more effective than others. 
Ns example, do administrators accede to particular parents’ requests be- 
cause chevy view those Pareys as having some coercive power? Do school 
“8 : psychologists carfy more power than other members in the diagnostic or 
placement teams because they are viewed as having \‘expert" status? Do’ 
decision makers make differential decisions about families and children ee oe 
Whom they. view as similar to themselves? Social power analyses may shed 


‘ 


some light on the nature of the decision team interactions which result 


av 


in decisions about children. 


The three approaches or models I have mentioned are examples of ways 


if - 


. ee 7 
of conceptualizing this interesting research area. While I consider these 
models potentially useful, thete are clearly.a number of others which 

: provid Ngoverful direction and. organization to the research. ‘The point 


Ne . 7 ‘ 
Fis be emphasized is that an explicitly stated conceptuahization is needed - 


io) - . . + He . i 
| | a oe Vag 0 he | 3 
. 7 fs. -% , rs ‘4 y * : : | 
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in order,to direct the research activities and to integrate and Anterpret 


the findings of the various studies.. More importantly, perhaps, the 
P Way & 


s 


concéptualization will provide the bridge between the studfes of assess- 
* 7 ‘ 


ment and decision making. It is this linkage which is the key to dppro- ‘ i 
priate services for LD children. : ihe Minn euded meses is’ to be con-. 


@ 


gratulated for taking on this complex probiem. 


- 
e : 
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~ | a Libby Goodin | 
. - Philadelphia Public Schools 
é x, os 


I have reacted to the paper. as a consumer, probably hdcause of myo 
| mm a 
ties to.public school programming. As I“was reading, I was looking for 


‘some indications of how the ‘research at the Institute for Research on Learn- 


¢ 
a 


ing Disabilities will help me deal with learning disabled youngsters I 


r believe my consumer perspective is co ¥gtent with the charge, given to the 


Yesearch institutes, to become involved-in basic research that would-ulti- 


4 


mately lead to practical applieationss. othe questions that Ysseldyke and 
his colleagues. have posed, related to the Ltakage between Besesenene and de- 
cision making, have accomplished thole ideale. The eevee. coaniienant Be tee 
weakigate the decision-making process, savticaiariy within: the context of 


the ‘multiple disciplinary team, was most appropriate and very welcome to me 
v 2 o 
personally. At the public school level, the team meeting has become a cri- 


tically important point in the total process of referral, identification and 
‘' placement. Individual professionals who once held sway over the identifica-«. 


thon and placement off learning disabled children, ‘such as the psychologists, 


v 


or’ the coordinator of the LD program, ng longer have thé power, if the team 


* functions as it is supposed to, to arbitrarily or uriilaterally make decisions 
‘that affect children's lives. The decisions which impact on our: youngsters 
* w , . 
ie 
- are the outcome of a mujtidisciplinary process and no longer are the sole a 
’ My a e 


prerogative of professionals alone. Parents and their advocates are also 
part of the team if they wish to be. 


.Unfortuhately, some professionals resent the loss of that power. 


Others resent-what they consider to be an interference. Perhaps some are . t 


. 


annoyed by the added demands that are now made upon their time. I am hope- ; 2 
\ . 


ful that all school people will come to realize that, in this day of account- 


> 


ability and legalities, cbllectiv§ responsibility is indeed a very good thing © 


¢ 


. - ¢ ? 
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~ 


Personally, T welcome it. It i important ae PancEte also realize that 
their pattteipation in the process carries with | it a share of the responsi- 
btlity. The placement decisions that%ere made, thé program that is hammered se 


out and dpcumented,in the IEP, represent the work, the wishes ofeall parti- 


cipants; we, the pbotessionala and the’ pardlits, must share the glory or the 


blame. The functioning of the team has inobicacions Por the, whole school 


s. 


. ~ a 
“ system. inketalays it, exposes us to public view and to public censure if ,, 


that is necessary: We ‘have to a our profess tonaliam, our judgments, and ° 


¢ 


. 


quar Wiigiees on the table. And the ipueiie can say, "Yes, we Like that" 


_ 


or "No ,. we don't like that." If the mul tidiseiplinary team functions well, 
: “¢ ¥ a 
the educational process proceeds as it should. If .it doésn't, procedural 


safeguards come, into play, children are inappropriately placed, educational 


a 


exocrads are interrupted, confrontations emerge, legal aiotateevae to , : 
the cites, and so forth. The work of the Institute relating és the | 
factors that contribute to decision matehiies the distinct roles of the mem- 
‘bers of the multidisciplinary teams, and so on, is going to be extremely ~ 


valuable to us. The questions that you pose, certainly hint at the many 
practical applications; I urge you to proceed ae quickly as possible,” I'm 


very anxfolis to see the results and I have a ee that’ they are going to 


. 


be extremely helpful. 


You mentioned the need to distinguish learning disabled from other ‘ 
iN i , i 3, ~ 


types of handicapped. You lee with es that we not digress into a 


lengthy discussion of the definition of "learning disability." I do not 


+ . a, 


 ) : , 
want to get into the middle of «that issue for it has no end. However, I 
feel compelled to-say that I am’concerned that the planned investigations, 
os ~ 2 ‘ 


oF particularly those that relate to characteristics of learning disabled - 


v 


‘ . 
— 
fa 2 . . ’ 


§ | a Sie 
,, populations, may add little to our understanding, oF the nature of learning ‘ 


disabilities. Much ot the proposed. research rests heavily a the use of 
many of the tests that you berated for problems of reliability, validity, 


. ~~ 
norming, and _80 forth. I wonder how ‘much more'we will know about the nature ~ 


& > 


of Jearniis disabilities from using such instruments. muse gee of defini- 


tion cannot be ignored. “a am not proposing that “the Inféttute undertake 
‘ . . be 
to put forth a new deeinieion, “but I ‘would’ urge *you to consider ,the possibil- 


a y _ , . » * 
ity of expanding the scope of some of yofir investigations in.order to delve © 


into the identifying.and distinguishing characteristics of learning disabled 
: : a ae , 

children. _ 7 se s © 

Reference was made to the new Roe ee kone on learning disabilities. 


‘ ‘ 


‘I suspect that\ many of’ us & site room realize that the new regulations 


really do not solve the underlying problems of definition and criteria. In 


o 
e 


one Reseanen area, an operational definition is isha to identify the léarning | 
ie , 


2 th 


disabled epudene® that would be the target aa Essentially, it Ry 
tM Vad ° 
, stated Ehat the children would be‘ functioning .two years below capabtlity or 


~ 


\ pennants this ivan underachievement model. Tats apEOas ls which depends 


a 


merely on a ie cine between BobreveMment. and a lat will not do. 


co 


This..operational definition will very aateely disintegrate within the con- + 


e 
. * t ” 


text of an urban school, and ‘under the peossuuesoe a urban environment. 
By this Seanaaed: as many as 60 er of-the children of the egtire 
: | ‘ school age population in certain subdieebiens of Philadelphia would be 
‘Learning disabled. The learning disabilities catdgory, particularly in , 
large aa a8 abused and is subject to overinclusion. * The abube of the 
ee Ibe! is amplified in an urban setting. Learning disabilities ‘often is 
| confused with duventte Asitanaeney Yes, there is jasentea disability and, 2 


wt 


juvenile delfquency overlap, but not a one to one correspondence. Learning ™ 
é od + . . , : ; * : 2 


. ‘ 
create’ ; He * 


' . a 


“. ¢ 
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disabilities is confused with compensatory education. Learning disabilities 


is now seen ~~ ee. me to EMR elased fleation: daestnent must 7 


Neto us to distinguish the truly learning disabled from the larger mass . 


of underachieving children. ‘This issue cannot be Tattered | or set aside, 
g - ‘ 
I iperee with you that a prevalent approach in ‘the past was ta, hOets on 


identibivation ‘of ‘the cause, followed by none Miatisa of the inderiiant pale’: 
. f im 


and finally, eonsdiation of skill deficits - a) cause, b) remediation a£ 


. on ee remediation of aadeate propsenss The peaules of such “yO 


tnvestigations have been ‘generally dtsappointingy But od question whethar ' 
that ca sequence is unalterable, Does the ea redeten of the cause 


RESePEsrtty dictate an effort at its remediation? This course of action 


‘ a 


would certainly be questionable if the causative factors .are biologidally 


bawed Have we adequately researched under lying causes? I ask this question 
ww : " . 
_ because L suspect that perhaps some of the differentiating factors: or char- ' 
acteristics we are looking for lie in an investigation of the etiological 


- ‘ 


factors and the biological bases of learning disabilities. Remediation of 


4 , 


: ’ 


_ skill deficits, the last part of the a-b-c chain, has certainly been the 


° 


most productive. It has given us immedidte results and many elasstoon _ap- 


= 1 


plications. The researches and educators in that area have given us a 


technology that: apples’ to. al childrén with learning problems. And, it is * 
@ BY 
a techno}ogy ie addresses the issue of management of Padeaiees srebidtis 
% 
more.so than the identification of — disabilities. Note that I draw 


a distinction between the. identification of the child*and management of the 
learning problem, : 
1 . te 


One “reference in your paper stated that ere disabilities is not a 


label whieh appiten. to some children, but eaenes a label which applies to 


a. ee 


. BF ¢ . P ad - . Fs 
all children and t at the goal may well be to develop an exemplary instruc- 


’ 


o 


‘ tional, technolo which is applicable across all exceptionalities.” This 
. 3 ; og : , , rY 
wis'one alterwative:. essentially, the &bandonment of the |LRxeategory as 


a distinct category of exceptionality. Although I personally believe that 

* ; a oN ai ; 
Were are truly learning disabled youngsters, I would prefer this alternative ; 
to the present state of.affairs and the confusion that ‘still exists in our, 


field. Unless. we confront the pyoblem of distinguishing learning disabled ¢ 


4" x 


> youn ers from- the non-handicapped and the further problem of distingmishing 


. the learning disabled from other categories of handicapped, I ant afraid that 


o it 


. this will be the only alternative and the ultimate outcome. Jim, the title 
. . Cae : ma ” <. . . 
of your paper was "Psychoeducat ional Assessment and Decision Making." I 


note that it-excluded "for the learnjng digabled." | Was this a premonition. , 
; ed ; ; ; 


oe ¢ 


, 


of the future?. . f 


(¢) 
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@irects what it is that we're talking about. 
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- Papehoedadietodal Assessmeny and Decision —_ 


8 ; mah Reaction - 7. ~~ ‘ 
; Rickard Woodcock 
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Measurement/Learning/Consultants: 
4 | ing 


= 


we 
i? ° 


I'm going to assume the role ef the gadfly. My comments don't 
he , 


iv 
‘ ‘ ¥ 
/ x 


concern major matters. I didn't find any. . First of aul. I want to 


o 
. 


talk about three of the terms that were used e your paper. I think 


t 


that in the field of measurement and its application to learning disa- 


a there axe miscgnceptions about thdse terms which may mislead 


our thinking. One term I'm concerned about is used when talking, about 


‘Mietalieeeaended tests" and "eriterion-neferenced tests." It is.the 


. . 


word test that I'm concerned about. It seems to me that the term mis- 


a 


- «criterion nete@renced gr norm referenced, bud Sweheedettatn forms of 
interpretation that we'can apply ES the results tha@t.we-get from a ‘test. 
The same test results foi a certain. test sould be interpreted, either way, 
or a test ‘ehat was designed to serve one kind of assessment purpose might 


_very well be. used to serve some other sort 2S ai purpose, It 


s 


@hould be made clear that it is the interpretation we're talking about, 


v 


not tests. 


I am also concerned about the use of the word "standardization" in 


oo 


zation”: “as being synonymous with "nonetn.” Norming is part of the pro~ 


cess of standardization. Norming, may or may not be an essential ingre- 


. 


Te mg 


‘ 


‘that ‘the test will be administered , scored, and BRST pretee in the same 


. 


Way. by different people: To insure’ consistent jaterstetation in norm 


’ referenced diteppretations, eablies will be needed. Norming is the way 


to obtain the numbers for ‘those tables. \ 


a 
oi 


It is not the’ test that is. 


\. 


_ the naper as well as! in the fielt. I do not think of the word "standardi- 


qs 


dient in that process. Standardization, to mes _- the iideeus of insuring 
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. 


The third term that concerns me ts "reliability." When we talk 
” iw : *: 5 ; ‘ 
‘ about reliability, we are talking about the precision of scores obtained 


. 


, From tests. I believe that Reople in the field assume that the. two - 

" statistics Veber oued eeeur of measurement" and "reliability coefficient," 
are Aaat Se different facets of the same ‘thing. I don't believe they 

"are at lall. Basically, we havestwo different kinds of séotes that one 

"Bit of tests. We Kaye scores that reflect the enone of an ability that. = 
7 person has; these are called "faw scores" or "grade suuivalant sneer : 


a, ‘a be "age equivalent scores." With the advent of the Rapich model, we are 
oe now eile es talk about some other kinds of scores chat seem to have su- 
P i Se ren ee ee or mathematically. We can cath 
about Rasch ability scores fox they are scores that represent the amount 
.of something that a person holds. The standard error of measurement 1s 


at the statistic that tells you the precision for that type of score. Another 


o™N class of scores that we deal with are scores that reflect standing in a 


oP 
¢ . ~ 


sane Gacten of people. These are the percentile ranks, standard scores, 

‘ . _ and the ae NCEs. The index of ‘precision for that ede Gi score is the 
reliability soahPietent: The standard error of measurement and reliability, 
statiscies ae not atenchemeat ie eyenetiounh many people in the field 
assume that they ‘are. When we talk about single seneces we talk about ena . 
standard error sinneceubenene of a eis ong score or the reliability of 
scores that come from that test. When we evaluate the precision of. 

; . difference, then it is a standard error’ of eae ar ihe difference 
‘ ‘aeore-that is the appropriate statistic, not the reliability of the dif- 
ferences. The reliability of the difeetence score te weed when the 

eqncert is whether there fave change in standings of a person. And SO, 
e  4 . _., $m, the field we frequently run.onto statements to thé effect that, we 
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- 


shouldn't use a parfcular procedure when you're .comparing two test 


scores because the reliability of the difference 1s low. ” 


You -mentioned the importance ‘of having tests with aeewnee relia~ 
bility, with a common criterjon of’ adequate being 90° or higher, for indi ~ TON ea 


‘vidual decigions. What if you don't ‘Have a test. with feliapintey that: good? * 


You still need to make a decision. Don't you aeil1 wane to- make use a tha, 
best information that'ss available to you? The feos oe éaieae: has’ ee 
appreciate the fact that ‘there is much more ‘chance imprecision in those 
date being used, to -form a decision. Also, Jt dis inappropriate to compare ee, t 
reliability coefficients for beets when those reliability coefficients are 
taken from different studies. The reliability coefficient is a function of - 
the standard error of measurement: and ‘the standard deviation rae sample 

. to whom the test was administered. ‘Thus, ae coefficient will ‘change de~ 
pending upon the sample tested. ‘One of the things that ce need in the ei 
is compares ve reliability coefficients, where the same tests are given to 
tha same subjects.. The. Institute might easily collect these ie some of your 


planned research. i 


In regard to the planned studies, I have a few brief commants. I do 


"norm-referenced" is used in Research Area l, 


not understand wity rie term 
where you are concerned with the’ adequacy of "norm-referenced" data for the 
prediction of success ina highly systematic instructional program. You 
asia be using taw scores or other types of secre. It is the content of . 
those tests that is important. 

My sonene on the study in Research Area 2, where you use a computer 
to look at technically adequate devices in assessment and decisipn making 


is related to i aaa point: when you are asking people about tests ¢ 


that have sufficient reliability, tage a more appropriate question would 


: ’ 


~ fon. 
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w an ‘ * 
be "Which tests have the best reliability for a given purpose?" 


Af 
at 


Reséarch Area.3 involves the comparative research on: children who . 
are failing academically and .labeled LD with those who are not labeled 


LD. I would recomménd using the independent Scholastic Aptitude clusters 
‘ r) di 1 a 


as the measure of ability rather than the Broad Cognitive Ability cluster | 


“of the Woodcock-Johnson Psycho-Educational Battery. Each of the Scho- 


o " e 
lastic Aptitude clusters is a special "intelligence" test. Also, ‘when 


NL 


you are measuring written expression, you might want to include the Proofing 


;subtest as well as the Dictation subtest. 


me 


\ 


: ge 
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Follow-up Comments to Keactants i 


James E. Ysseldyke 
University of Minnesota 


© - « \ . s 
I want to thank the Beopts ene have provided ‘very insightful 
: ® 
wpactions to the paper . We'd: ear hee neve people tell us now those 


_ aspects of gur ‘research . that we gacuid ‘be concerned - ahout than to tell «~ 


& Hl 


us five years from now. There are some overriding concerns and iigeues 


th 
that we find ourselves dealing with in srobably ine same way that. the : 
reactants do. * ~" = Ye : ‘ : MS og 
, : , Pe 
I think all the reactants raised some ‘esnse@bagardine the » 


4: w. 


A , 
language we use in the field’ of letting disabilities. Terminology is. 


"4 . 
‘ 


- important. There ig a tremendous, variabglity cceu Location to location 
in terms of ‘the.k mds of children we are talking about. The Chicago 


Institute has e essed ‘their large concern with definitions for chil- — 


‘ 


drew in both — an and inner city Settings. But, it ig a terribly 


L {mportant issue for us also. . , — 


Dick's comments regarding differentiating between interpretation 
, 
» and the norm-referenced/criterion-refgrenced testing is important for 


us to-deal with. “I think our approactt agrees with your recommendations 
for interpretimg data: any test is merely a sample of behavior. 


Two things that I, didn't want, to miss responding to on Dick's 


concerns were the notion of the reFtabilitites that were recorded in 


the tables and the fact that the rates are there because they differ 


fe] 


when they're standardized on different populations. The data that are . we 


reported in those tables are taken directly from the manuals and are 


" the standardization data that the authors themselves provide. The 


~ 


rarfges reflect simply the fact that reliability changes as a function 


. 


(¢) 
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_we just. don't have a reliable and. valid means of geting at a particular ‘ 


of the age level of the individuals involved. I have trouble dealing 


y 


with the notion that a .90 criterion might be unrealistic and that we 
should use the best available device. The best available device, in 


ii sla a terribly low reliability. Decisions are made on the 


4 


basis of unreliable data. I think that we ought to be very honest about - 
what we are doing and say to the parents and teachers:and fo others that 


ta 


concern. : 
- ; > 7 . 
- Barbara's commengs on having a decision-making model ‘to guide our 


4 _ 


efforts are extremely helpful. Although I agree, we are having trouble 


4 ) 8 2 - . 7 
finding relevant models. The heuristics notions of Shavelson and 


' « : ‘ @ . 
Kahneman andotheys make sense but we have had some difficalty seeing 


» 


how they make sense specifically in the placement team context. We 


. 


* ‘ a A 
have not been able, at this point, to identify a mpdel to use to analyze ‘ 
( 2 : 
what goes on. Instead, probably the model-.we are using to analyze. what 


goes on is just straight-out naturalistic observation in the situation. 


. r . 
At.some point in the future, we will attempt to relate our abservations 


to existing models. «, . ~~ 
“ x - 
: y ; ; as 


r 4 


“te 


an 


Hightights of the Roundtable Discussion ‘ 
: Following Ysseldyke Presentation 


-a eo * 


Discussion began with several Roundtable members expressing defi- 
nitional concerns. Questions were asked regarding the nature of be- 


haviors the Institute researchers would use’ as both marker variables, 
p , | . 


for the purpose of ‘describing subjects, and as definitional variables, @. 
a the purpose of defining children as learning disabled. Alternative 
definitions were described and discussed, ranging from defiattions based 


-entirely preeehoet labeling practices to! test—based definitions. It was 


obserwed that much of the research being conducted by Institute personnel 


- oe a 


is nét aenandent on defining a deel of bnilaren’ as ies qtae. dicaBied: 
owaver, the results of several. Institute research studies should shed 
light on the important definitional problem, 


.. The strategy being es by the five Institutes to deal with the 
ay 


definitional issue will be to describe the populati< ad they 


* ®.. 


work on the basis of the same variables, such as age\ grade, SES, and - \ 
Py : ‘ 


~y measures of ability, achievement, and interest. The Institutes will use 


the Woodcock-Johnson P&Sycho-Educational Battery to obtain the latter 
measures. this way, identical descriptive data will be gathered and 
the results compared. In the long ‘run, the Institutes will dttempt to e 


_ make statements of the relevance of gpetific research findings. to a 
variety of settings. . 5 : — ‘. 

4. The marker wertayee study betiig, cohducted by Barbara Keogh at UCLA 

also was cited as an impgrtant step cewek clarifying the necessary 


components in descriptions:of the research population. In its first 


year, the marker variable project conducted a comprehensive survey of 
. , “we 
; 


. 54 


e 


‘ 
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+ ) 


\ ‘ e 


i ; : 
the research literature and found inconsistent and non-comparable char- 


o 


acteristics being used to describe learning disabled populations. ‘It was 
- . : J 
noted ghat, given the dismal situation related to definition in past, 


,and current research, any attempt to determine whether youngsters in 


i] “. 
the research samples of the five Institutes are similar would be an 


t 


~ 


enormous step. : é 
‘ 4 . 


( 


Roun¢table discussion continued with consideration of the issue of 


what to assess. Specifically, the distinction between the ability/ 
. + 
process orientation to assessment and the skills orientation to assess~- 


ment was questioned. It was noted that the two approaches have differed 


within special education. On one hand, the concern has been to look for. 


\ . 
difficulties within a child, in terms of information processing variables, 


psycholinguistic abilities, perceptual abilities, and so on. Within 


{ 


special education, this ability/process orientation has attributed the 


failure of a child to acquire academic skills to causes within the ; 

| . . 
child. A contrasting approach denies the value of searching for within- ’ 
child causes and turns to a more curriculum-based task analytic prq- 


cedure. This skills orientation has sought to identify the child's 


skill level and then work from there to deyelop higher level skills. 
. « ‘ : . 
in a sequential manner. While there has been considerable empirical 


support for. the task-analytic skill-based approach to {ntervention, 


there has been little empirical oy aan the" payoff from ability/ 


- progess assessment and ability traidqing intervention. ‘ : 


It was suggested that the distinction between the ability and skills 


t 


, approaches might be an historical one rather than a logical one. The 
7 x . 
two approaches cannot be distinguished on the basis:of their focus on ° 
cognition (since the skills approach ultimately is tied to cognition 


4 


2 


v 
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ene in the way — it task-analyzes skills). Rather, one approach 

did not seem to work (the ability approach), so another approach that 

did seem to work (the skills approach) was adopted. This does not mean “ 

that the ability approach cannot work, but perhaps more realistically, ° ¢ 
that it has not been Seabed adequately. It was’ observed that individuals 

who have attempted to train processes have used skills as their dependent 
measures, but generally, ave not tavght these processes (e.g., perceptual- 


motor processes) in the specific domain ‘to be tested (e.g., reading). , 


Teaching processes within specific domains might lead to different results 


than previously reported. \ a8 
Another issue raised was related to the suggestion that decisions _ hee 
should be deferred when only technically inadequate informat ig was avail- . 


able to a decision-making team. dt was argued that, given the current’: 
functioning of multi-disciplinary teams and the different levels of de- 
cision making, decisions cannot be held in abeyance until adequate test 
(nebrumenits are available. 7”) ( thi, 

pre was agreed that we are forced to make decisions; we. cannot =| 
for a moratorium én Ndectaign making in senieal settings. However, \ 
the problem with using the "best" available tests in most cases is that 
we cannot eenneters that the measures we select are indeed measuring 


*¢ 


) . ie - 
what they purport to measure. One solution is to have alternative means 
7 : ‘ 


of data collection which are not as dead particularly for making 


instructional intervention decisions. It was noted that the best way 


? ; 
to determine what content to presdyt a child and what instructional 


Methodology to use is to teach the child. 


Decision-making teams, however, must make, decisions related to 


“57.000 


¢ . 
They must maké decisions regarding 


g. 
te 


other .than instructional concerns. 
“Gplstondnes related services needed, parent involvegent, and so on. 
> make 


The ice tOn of what sources of information should be used.t 
’ 


“or 


these kinds of decisions was faiged 
It was suggested that placement decisions need to be test-based 


decisions, but that documentation of ‘the need for a specific placément 


must be made as well. Considerable data should be gathered on the al+ 
ternative instructional techniques and strategies that have been attempted 
in a youngster’ s educational program, ‘and ee ‘they succeeded or failed. 

4 


In some cases, decision makers will simply “have to admit that they do not 
e 


ieee ‘adequate means for making a decision. 
ge interactive nature of desesshede and intervention was then 
It was suggested that assessment which penties in identification 

Special education 


‘ 


< 

dtacuseeds. 

and classification clearly has an effect on instruction. 
eeeieing programs provide different courses in curriculum and in methods 
for children given different labels, such as mentally retarded, ae 
disturbed, learning disabled, autistic, physically handicapped, and 80 on 
Aiehourh not necessarily am approach to be advocated, the classification 

/ 


procedure purports to-tell us how to teach 
However, the claim that assessment and teaching are interactive { 
s 


7 i 
implies a two-way street Behavior analysis is an excellent example! 
of the traffic flow where assessment influences intervention But / 


what evidence is there® that the traffic flows on the other side of the 
é . 
) 


o_ 
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Ve 


child gets tested? 


". gtreet, that intervention influences assessment? pele 


- ou ba 


does what happens to a child in*the classroom influence how that a 
* . % x, 


Pt 


‘ + ? re 


Several responses were made to this question. At a gross level, 

- \ F e . * 

a teacher referral indicates that the child's educational program has 
in some -série: been inadequate; this event sets into motion a series 


> : 
of assessment activities. ? Those who are working with children ought 


‘to gather data on the intervention process.: Tae faterventtons tried. 


should dictate the data to be collected. Intervention and assessment 


would then go hand-in-hand, 
J & ae ‘ ’ 
The concern was raised that while continuous gathering of data | 


may occur prior ‘to a decision, once a child is placed; the decision 


'sseems to be chiseled in stone. Data#collection is not continued once 


& - 


a child ia paced Assessment has not received the emphasis it should 


. 


have in re-evaluating placement decisions. Once made, placement de- 


b 


cisions should be considered as tentative, subject, to further input 
; 


from other data gatherers. 


y 
y 
It was suggested that while the idea of rethinking placemert 


“ 


decisions is important, the present team decision-making process} seems 


to, inhibit its occurrené. The placement process is a cumbersome one | 


¢ 


, whicw@#verburdens the individuals involved. However, the system, does 


have a built in annual review requirement, with re-evaluation reguired 


every other year. The latter requirement should fokce educators td 
. e , 


of the youngster. Unfortunately, the system is difficult to work within 


and typically is very slow moving, even when there is information | 


. 


look at the appropriateness of the continued placement ar categorization 


n 


g 
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which dictates that another decision should be mate. - . 


: Rs 
* eo: 
-t ; . é ' 


Lt was noted that’ data collected through standardized tests some- 


~~ 


- how area valued more highly in decision: making than data collected 


re 5 . ey ‘ , ; ; . 7 a 
s through direct observation by teachers in a more informal and unstandardized 


so way. This happens even thou the unstandardized, direct obseryation — 
) . . 


data could be fed back into: the decision-making process more regularly. 
Such biases may be a reason for the apparent greater influence of a, 
‘school seyeholoxist 8 opinion in ‘decision making than the opinion of ° L 


the eet depts worker with the child. ‘ 
One sapect of the Institute's ceceanch is eee Siaay? rtvegtis 
, \ ' 
such interactions and EE eeneee in. secpane teams. searchers also . ” : 
will be ee the extent to which intervention data ae yaiy are ‘ . OR 
; input into the “decision-making process. Although faecrasatl ca data eg 


ra aay . . 


/ ought to be importaht in making decisions about placement, the extent 


to which this actually, occurs is questionable. 


, 


‘ he aesue wag then taised as to where the Institute's research on 
‘ ao sincenene-teen decision making would lead. It was suggested that the 
_ findings should somehow lead to an improvement in the decision-making 
process. “However, there is a problem in that there does not seem to be 


a ereer variable: we do not know when a decisioh is right and when 


er it is wrong. 
a Came 
‘Fy . It was sippenticd rhe the donendaak variable data will probably 


. ‘ e . 


have to come from pupil outcomes. How the pupil does will determine 


e 


whether a decision was right or wrong. The result of the research 


should have great potential for influencing the process. At the present 


time, the research’ concern is to describe what haSpens. * Such a* ‘ 
. . 3’ 
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eet d 
s 
. 


description should teak to some ‘statements abdut, desirable icesion. 
om 
making. In the future, the Institute will probably be engaged in ~* 


specific Receaveh to inf luence ‘the ‘decision-making ‘process. one” ap- 


proach would veo compare and contrast alternative decision-making: 7 


‘models. Another approach woytd be to. train placement ‘teams in decision 


making and look at the influence of such traning. The’ present. des- 


s . > 


criptive study might indicate that people spend 90 percent of their time 


te ' ss * / 


in deciston- ng meetings presenting data. Given ‘this finding, it 


might be’ possible to prediét ‘with 90 percent accuracy.‘what the decision 


would be if the'data were available ahead of time. ‘Thus, a third ‘ap- ‘ 


re might be to enter all data into a sompater, give the decision-making 


s y . 


team the computer-based deciaiea. one then evaluate the inf iuence of 


- 


this informatior on the team's dedision.. While, there are a great number 
Pa “ b 


of posssbilities: for influencing Ge decigion-making Pines empirical 
: ¢ 
evidence is needed Fano on what currently happens yin the entiive as~ = 


t 


sessment and decision-making process. 


ar aan 


a, : Stes ¢ 
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: » 4% . a eaten . 
.ducted at the Institute. It was noted that the direction of the research 


be * . of 
‘ ‘ . ; . > 


‘ “Highiights of the Open Discussion " te -% 
| “Following Yeseldyke. Presentation Ye 
bY 
| The Open discussion began. with comments on the research to.be con- _ 


4 


‘seets ‘td be to identify some general principhes, ones that can be used ef- 


fectively in applied settings. It was suggested also that despite their 


commendable emphasis on ‘the’ non-testing aspects of the assessment process, 


a 


the researchers should realize that observational methodologies, as well as 


- interviewing, are not simple cudp-alls. Many problems will be encountered 


in attempt ing to use an ecological «approach. Another: comment dealt with 


rs 4 


the notion of the muLtidisetplinary team. icnouss called multidisetplinary," 


ole 


team members really,are very ah ‘alike. One mnt se cmuiees that - other 


individuals were included, individuals who traditionally have not aes in- 


volved in the psychoeducational system, the decision-making process might 


be very different. - 7 a 2 7 

"the eerie of considering the weight that a person assigns és 
a given piece of information when making educational decisions was then 
raised. rt example, a person might be eiven a great dea of empirical 


data yet ‘reject it because he or she has a mental model established about 


. 


what is tmpottant in identifying a learning disability. “Will there be 
any attempt to detetmine the weight of a papéon's Gental mores and the 
extent of empirical data needed to override che model? . 

- ae alae it -was agreed that one's moder or sek does influence 


decision making. The simulated decision-making research will address 
this igsue to gome extent. It will allow the researchers to introduce 


“* 


certain data (such, as sex, SES,, and oo information), analyze ° 


es effect on decision making, and then compare these results to data ' 


obtained in studjes in which decision makers repott the extent to which 


» . ‘ sale 
quch information influences their decisions. It might be predicted 


oo Z 4 “ey 


‘ ve * 
. : 
Cy - 
” ; 

, 


a 


62° 
that decision makers say the information does, not influence their decisions ° 


. ~ a ae 
B even though research findings typically indicate that they.do. Research” © 


on attractiveness bias, for example, has indicated that a child's looks 


do not just create an initial impression, but rather! influence grading 


and other educational events that occur throughout the school year. . It 


tay . 


was suggested that because such internal influences clearly exist, it 
is gmportant to build them into the system to be recognized and studied. 
The point was then made that such simulated bogus study information 


can have restricted value becatise only one variable is studied at a time. 


4 


The’ decision makers are ia information that varies on just one variable. 


It was suggested that if @ decision maker is presented ‘all data, then a : 
) “— 


s 


factor. that otherwise might appear to have.a significant effect would be 


washed out. ; ° - x 
L 
. - ‘ 
o 


In résponse to this, it was noted that the research iii 
just one variable. Even in practical situations (the classroom), whete 
‘all variables were operating, a variable like attractiveness still was 


: operating. It did not wash out. It was suggested, however, that new ~ : » 
a r) . © _ 
; information does serve to modify one's previous model. Weights are 


4 


put onto new bits of information and these weights determine the influ- 


‘ 


ence on one's model. 


Another diddienes member suggested that eet important™ factors 
_ have been iignotad in research en decision making. These ara: factors ace 
exist out in the real world of sducatitton - declining enrollment; fear of 
loss of jobs, and other irrational variables that do enter into decision 
cr ' 


$ - yon ‘ 
making. Lt was agreed that such vartabies certainly are important and 


—& 
that the research will attempt to look at them. i 


4) . : , , ‘ yo 
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63 
The question was next raised regarding the extent to. which ihe 
Institute wound be fnventipat we the re-evaluation process. Once a 
child is. a are needed changes perceived, and if ipaxeotved, ‘do they 
actually occur? It was natal that although such igwaneieaciene are not 
axpitertiy included in the proposed ceseaeehs they certainly will be ¥ 
‘ , “area of ‘concern’ for subsequent research. Studies would involve deter- 
mining the extent to atch data on a youngster's success, in : variety 


= of interventions, are used in the process of making placement or continua- 


tion of placement decisions. , ., 


. The discussion concluded by shifting back to the definitional issue. . 
~ , 
It was suggested that the treatment of a learning disability might be 


I . greatly improved if its etiology were knowns The situation might he con- 


4 


\ sidered similar to that in the area's of suicide and fevets. Suicide is . 


‘ - : 


a behdavtoral symptom. Initial research which.attempted to corr¢glate 


? 


Various factors with the Stcurrence of suicide was not very helpful in 


Seaetne tee occurrence. The abieiseys akes a big difference in treat~ 

ment. Similarly, fever is a behavioral symptom; but, treatment raiiies 

gn knowledge gt cbiolioay: _Learning might be considered in the same way, 

as a behavioral Synpkon-ehat needs to be treated with consideration being 

given to etiology. [t was argued, with dinevoicabeaneae. that whereas 

the dependent measure in suicide ot fever je quite reliance. the dependent 2 
measure in the identification of jena disabilities is not adice - 


reliable. , a 


In relation to the definitional issue, it-was noted that the Institute 


- 


. 
- 


plans to look at learning disabled students and students’ who are failing 
. Ca . 


‘academically but not labeled “learning disabled." Data on these children 
y Bee 


‘<) 
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; oo ‘ : 
will be collected,gthen presented to diagnosticians to see whether they t 
of ¢ » 
can identify those children already labeled as learning disabled. If ~- 
: —t—- . 


¢ 


*thds regearch indicates that diagnosticians cannot agree, the field 


really has problems. Furthermore, researchers are faced with a di Jemma ,’ 


for it will be inpaéetbie to reliably identify who should be ee 


= 


as learning disabled. - 
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Behavioral Research Methodology as a Basis for the Formative 
3 Evaluation of Learning Disability Service » 
Stanley L. Deno 
“University of Minnesota 7 8 


¢ 


; . “ 4 ¢ 
CPZ schon the contributions that the behavioral approach might 


make to the assessment of children with learning disabilities, three general 


categories of assessment decisions were-considered. First, identification 


—— 


decisions (which include referral, screening, and classification); second, : 


program planning decisions (which include goal setting, curriculum and in- 


struction, and level of service); and finally, program effectiveness (which 


includes both formative and summative decisions). The two types of deci- 
-. & y 
sions in the program effectiveness set require some explanatio . Normative 


decisions are the decisions made by people during the course &f implementing 


! - ‘ 1 


' - Cc 
. program plans. Fermative decision making r@fuires answers to questions 
such as: "Do the methods and materials seem to be working?"; "Is the 


cs 


level of service, appropriate?"; "Are the parents satisfied?"; "Should we 


+ 


change something?" Formative decistons are made, then, to help improve - 
and’adjust a program on a continuous basis. Summative decisions are an 


outcome of judgments made either annually or upon program completion. . AZ, 


Summative decisions are after-the-fact decisions typically made by those 


an" 


persons empowered to certify that..a program sytceeded. The essential ' 
summative question is: "Did the program succeed?" An implied question 


is: "Should we allocate special education resources similarly in“deve]y 


.»y ; i 


< : pany 


oping future programs?" 


While a Anseteene for Research on Learning Disabilities has 


om 
taken on the task of understanding and {mproving assessment practices for 
.o , ; 


children considered learning disabled, this project's goal is to bring 


« ° 
, 
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: oe oS 
the behavior analysis perspective to bear on the problem of assessment 
: : } 


. 
* 


for oy all of making those three general sets of decisions. In doing so, 


we~have identified a set of relevant "perspectives," a few of which I am 
; ; é 
going to share wigh you here. ; 
First Perspective . , 
F The assassment of problem behauior should bé approached 


from the point of view that it evolves from and t 
. governed by the same laws that’ govern the development 
i _ Of normal behavior. : / 


. This first perspective deals with the assessment of problem behavior, 


% 2 


the kind of behavior that children must emit to be given a special label, * 
such as "learning disabled." The perspective implies that when a child's 
behavior differs significantly enough for that child to be eoreeea, one 
shout approach eesaxineiblas if the behavior to be assessed: can be accounted 
for withoy invoking a special set of determinants (variables) to explain 

the occurence of the problem behavior. The behavioral perspective is | 
‘that problem behavior, as well as normal behavior, isa cunetYn a hese 


sets of variables: (1) the genetic constitution of the a ee 


(2) the training or reinforcement history of the intividual, and (3) the. 


current environmental, circumstances of the individual. Often, people - 
: e . s g . 4 


Z { ; 
assume that behavioral psychologists think of the human organism as ‘be- 


ginning with:a tabula rasa; in other words, -that all human organisms are , » ’ 


~~ 


the same, without any constitutional variation. I do not. think that as- 
\ ; . ‘ 
sumption is representative for most behavio#al psychologists. From my ° ad 


own point of view, it is just as useful to aecume that there may be 
{ 

EET (A 

= These dre presented in Deno; S., Mirkin, P.K., & Shinn, M. % Behavioral 
Pdrspectives on the Assesgment of Learning Disabled Children. In J. E. 


Ysseldyke and F. Morrison (Eds.), Multiple perspectives on asse : 
of \learning disabled children, in press. , a 
f ca : & 
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genetic constitutional differences, but such differences aie probably 

the proper domain of phystoropiete and perhaps physiological psychologists. 
Behavioral psychologists typically do not study the iapience of an ae 
vidual's “genetic constitution. Instead, they focus on the historical and 


t " . 
current interactions between individuals and their environments in an attempt 


ee _ to account for changes in behavior. 
Just as people sometimes erroneously: conclude that behavioral psycho- 
{ 
logists do not believe in constitutional differences, I think people also 


sometimes assume that behavioral psychologists approach ‘the analysis of 
behavior as if there is no history -~- that all behavioral determinants - 
can be found in the current environment. I doubt seriously that any be~ 
. y 
havioral psychologist .would say that. Generally, the assumption is that 
; - a 
each individual carries with him or a a mass of experience which is , P 
. ' 


a described in terms of the reinforcement contingencies experienced by the 


n individual. That reinforcement history combined with constitutional 
. v ° ‘ . 


» 


the current environment. , i - oof %.. 


. 
. .- * 


‘variables, then, determines how the individual is likely to' respond ‘in 


4 me 45 . 
The interplay among variables of the genetic’ constitution, reinforcement ~ 


' ; 


history, and current environment was succinctly characterized by :Bijou : 
(1977) in an article that he published in Exceptional Children a couple 


* of years ago. He called the model "interactional," saying it consisted 


of two basic concepts. The firgt concept is that there is a continuous 


interaction between biologically devekoping children and the progressive 
yy . . ® 
changes in their environments. The second is that¢these interactiofs 


change the indtvjdual (that is, develop a person with a unique personality) 


on the one hand, and change the environment on the other. The individual 


. 
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acts on the environment and is in return infltenced by the eraee dae 
ae or hér actions bring abdut in the environment. } 

At this point it should be emphasized that while the behavioral model, 
its principles, and its procedures can significantly contribute to im- .’ 
proving learning disablities servigés, the focus of ‘the peesene Gancacch 

‘is not on the application of b , vioral principles to Learning ae ea 
Instead, the focus is on the research methodology used by behaviggal psycho~ 


logists to empirically analyze the behavioral effects of changés in 


individual's current’ environment. ip tener punishment, stimulus 


control, and schedules of reinforcemeht, are ppErnesples that ought to be . 
eta in developing learning disabilities programs. In fact, as I bvain. 
teachers, I require them to use those prjnciples in the developmént of 
int¥rventions. However, inPapproadhine the problem of assessing children 

with learning disabilities, we find more relevant the scientific methodology 
that behavioral psychologists have developed to experimentally analyze in- 
teractions between benaviog and the ehvironment. Thus, the application of 


behavioral research methodology to assessment of ¥%tudents called learning 


disabled is the direction of the present research. 


Second Perspective : 
nae ‘ 


The,methodology which has been developed to, accomplish be- 

havioral assessment is characterized by the careful des- 

cription of the behaviors of interest, the development 

of procedures for directly observing and recording that 

behavior over time, and the use of time series research 

designs to try to determine ‘functional relationships be- q 
tween changes in the environment and changes in individual 

behavior. : # 


‘The methodology is idiographic rather than nomothetic, and it may { 


é¢ 
be developed to make unique contributions to the assessment of LD studeyts. 


« . / 
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Time series research designs arg not the private acuta of behavioral : 
psychologists. One of my favorite illustrations of the app ioaeiin of time, 
series data analysis to look at human behavior serie from a Common Cause 
ceraieeees: The particular time series datum. collected was the number of 
pieces of franked mail that our legislators in gone con send home to ‘ 
their constituents over time. The question was, how much and when is 


franked mail used? The data are presented in Figure 1. 


* 


. As can easily be seen by inspecting the graph, a relationship apparently 
exists between when elections occur and how mach franked mail ‘is distributed. 
What you would, observe if this were a cumulative graph is the characteristic 
fixed interval souls an. An increase in use of: franked mailings occurs up i 
to the point that the elections are held, and then an immediate drop occurs 
in how much free mailings legislators use. The amount begins to increase 
/ | agaig over oe as an election approgches, and then it drops off. ; 


One of the nice things about time series data analysis is that it can 


& s 
eS "be used to keep our jpsislators accountable. More importantly for, us, how- 
\ a 
ever, is that {t cah be used in ongoing natural experiments. Careful ae «a. ~ 
sentations of time series experiments have been made by Glass, Willson, a 
.  & Gottman, (1975) and by Hersen and a pa good articles 


\ exist on time series analysis. An important feature of Glass, Willson, 


Ae . and Gottman's book, however, is that they point out the potential contri- { 
\ ° | 
bution that time series research design can make to the evaluation of: 


educational programs on a Loaeiiious basis. If you know what data might 


' be useful and significant, you can: then g&icinely collect those data, " 


’ ‘ 
identify naturally-occurring changes in the env{ronment, and then determine 


* 
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1 
” 


what relationships might exist between changes in the data and changes 


in the environment. ’ : 


Probably all of you have seen graphs like that in Figure 2. Most 
bd 


of the time they appear in technical reports written by behavioral psycho- 


logists. Figure 2, however, presents a graph (that a teacher developed to 


monitor the percentage of a class at was engaged in "disruptive noise." 


x 


Se Sa, press as a, nk nas 


hy 


a 
toy 


This graph shows variation in percentages of it seu alacisa across time, 
with vertical lines denoting points at which specific environmental 

changes occurred. The graph also denies a "reversal design," in which 

an attempt is made to replicate the effect: on behavior of changtag from 
baseline to the treatment condition. Revitestion: of course, is not always 
possible, but it is a way to eliminate plausible rival hypotheses for ex- 
plaining the behavior changes. In this example, there is a Necwetae in the 


\ * behavior both times the treatment aca Many graphs‘of this SON, 


type exist throughout the literature of applied behavior analysis. I | 


re 


have always been intrigued not only by the treatments but ans by the 
2 sobentital for evaluating individual special education programs whichis 
inherent in the research nethodology. Special ecucather programs are 
individual ly-ortented intervention programs. When looking’ at individuals 
Ny “oy instead of groups, a methodology ig needed to help one look at individuals 
to debe caine what influence a program is having on them. Conventional 
research methodology ‘typically Contrasts performance between groups rather 
than within indi viddale. When approaching the sedbieis of assessment 


within learning disabilities programs, then, the within-subject approach 


of behavioral psychology seems more useful for assessing treatment effects 


, ; a 2 
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-is relevant at this point. 


‘assessment af a child's behavior requires description not only of .current 


, 


71 
with individual learning disabled students A related perspective 
. aN *. 


¢ a 


a | ; 


Third Perspegtive . 


Behavioral assessment requires first, a precise description ‘ = 
of the level and direction of the behavior, and second, an 
empirical analysis of the variables in the current environ- 
ment which control the level and direction of that behavior. 
“>: & : 
Unlike most approachés to SCegeEnents the relevant data in the behavioral 
. . an] 


approach are not simply measures of the ciecen level of performance (or 
actiditeent quantified-as raw scores, grade equivajent scares, oF percentile 
scores). Just as important are data on the eee in which a behavior 
is changing (i.e., is it daereasine or decraasing?), since the behavior . 
may not be "4 While it is possible to determine level by assessing 


a behavior at a single moment in time, the behavior may, in fact, be 


e . 


. ' 
increasing. John Stephens (1967), in his book entitled The Process of 


Schooling, oer zes the fact that children are getting letter in basic 


s 


skills all the .time, and argues that the schools do very little to influence 
that: in any specific way. He hypothesizes a kind of general press that the 


school brings upon the child, to learn to read, write, spell, sakes 


: { 


Stephens' point that academic growth is continuous implies that 
. & 


* et 
level of performance at a moment in time, but also the trend or rate of 


y 


change in that performance over time. Rate af change is essential, 
a a ¢ 
t Le . arn Re! 
especially if one is gping to use assessment data to evdluate the effects 
, | . ; 
of services. If we agree with Stephens that children's academic per- 


formance is not stationary, observed changes in a student's achievement: 


might well be attributable to tonditions other than those arranged by the 


wv : , ‘4 
f 


special educators. . * 4 
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ordinarily obtain during initial assessment), that the intervention: line 


— eS 


‘i i ae 
Nowadays, special saueaelon is. charged with demonstrating .its . 


] 


efficacy to the consumer. Pre and posttesting J, practice to : 


provide evidence of program ef fectiveneds. Jt should now be obvious that 


pre and posttesting is ‘an inadequate apprpach to evaluating treatments. 


Here is an illustration from a class I teach. 


ee 
ay 
“ 


a ed 


1 
Assume that the little squiggles on each graph in Figure 3 repr@sent in-- 


2 


dividual academic pérformance data, and that the heavy vertical lines in 
: 


each case represent instructional intervention. It is possible that thé 


getrect of each intervention will be interpreted quite differently depending 


on whether one looks at level of performance alone or whether one considers 


F ‘ 


,level and direction together. To illustrate the problems of pre and post- 


. / 


testing to evaluate intervention effects, I propose to my students that 


f ‘ 
they interpret intervention effects first without the time series data and 


then with the time series data. I ask them to assume that the pretest 
datum point is that point immediately preceding the intervention (which we 
, -*~ 


¢ 


represents the onset of learning disabilities sérvice, and:that otr posttest 


‘datum is the last point’ on the graph (e.g., a year-end assessment). Such 


ituatioh probably characterizes much of the assessment: that goes on in 


, . 
evaluating services. One assesses where the Sraacde is, then sets up a 


program and teaches for a while, and finally, dsitetutnee whether the stu-~ 


v 


dent has increased his or her performance on a’ set of objectives, or has 
mastered a certain set of skills. If student performance changed from 
point A to point B, it is assumed that the intervention program produced 


the effect. As inspection of the time series data in each graph illustrates, . 


Ton) 


| | 13 
ee ) 


however, it is quite possible the conclusion will berdifferent if dhe 
has. available more information. In several instances, the growth from 
point A to point B is perfectly predictable Over a period of time based 


on the actual data from baseline. In those cases, no evidence is present 
; nd 
that the intervention had any effect whatsoever. In thet cases, while 
. : ; 7 
a "no effect" conclusion would be drawn from pre and posttesting, the 
complete time series would indicate otherwise. In the most embarrassing 
” "3 
cases, we might assume that oyr intervention has had a positive effect on «. 


~ 


- the student's behavior when inspection of the time series reveals that 


onset of learning disabilities services actually depressed or decelerated | . 


' 
- @ ‘ ’ 


the performance. oe — 
‘ 
As a seule of this eRenCTES the point is nee quite obvious to ~ 
the teacher in i aoaraa: Lf one really wants to be sure of the effects jp ' 
of instruction, one needs to.assess sepiounghes in auch a way that estimates oe 


fy 


.of trend, slope, or direction, as well as Tevel,’ may be obtained. That, 


4a: . 


I Believe, is an iMportant characteristic of the behavioral approach 


ad 


4) ’ 
to assessment, especially tn contrast to other approaches to assessment. 


« 


Fourth Perspective . . , . anes 
* The comprehensive behavioral teaching systems used'to. | . 


design temedial instructional programs differ a cea | 
with respect to assessment. 


Several, well-developed behavioral teaching systems exigt which can 
be adopted and used extensively in school programs. In the review of 
the data base, it was noted that chose appites wonacie at ayo eas differ 
with respect to the emphasis that they place on the eoliieetion of the kind | 
of ee series data that I've. been discussing. Some of those applied be- 4 


havioral systems tend to rely more on pretesting and posttesting aad may 


use post hoc teacher judgment on checklists as the primary assessment datum 


=> : : ee 4 
for deciding whether or not a studnt actually has a set of skft?Ps, a 


Py , . 


; P wt 
Other applied behavioral systems heavily emphasize daily data coltectioh. 


t 


A fain illustration of, those differences exfsts in the Distar. system, whit, 
é 2 ’ - i. 8 


s 
e 


oe to do ‘frequent pre and posttesting, and in Pretision coach ine where 
" 


daily measurement is the essence of the system. Owen White: (1977) ele 


+ 
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described the Distar developers as dpecescea in specific peeeieeiees 
%, ; — 
solutions to teaching care cular skills. In such an approach, -if ‘the 
"t : . . é 
a ‘ 
teacher has a student who is deficient in a. skill, that teacher. ts given 


a . . ° 
t . 


a + “ 


a teaching package which is, used to teach the ¢hild that skill. In. con~ 
trast, people in the Preehet on TEGCNANS tradition are given a packase Eon” 
creating time series eek anon the specific skill, and the teacher trained 


to use the daily timé series data to evaluate the effectiveness of instruc- 


tional SHREK VenETONS's packaged or ‘not. Since many vere instruc- 


tion are possible, the teacher - given the tools. to try to sppxaiae the. 


van 
i eae 
effects of different alternatives. : Oe 


¢ cad ' 


: . 
e ae v . . . oe a ° 


Fifth Perspective : * 


‘* 


The applied behavioral systems which contribute most . : : he 
uniquely to ‘the development of procedures for assessing $0 
children with learning disabilities include repeated p ¢ > 


aatey samp 1 ng and preeres displays of | time series 
da : 


, 7 re 


We believe that the behavioral systems which might make the most 


unique” contr button (rfot necessarity the greatest) to assessing children 
5 
with learn ng disabilities and to educational assessment generally, are 


om ' 


those whigh use or tee Repeated behavior somphine and graphic displays 
of time series a. This is not to’ gay that what faak teat and ‘other’ who 


a 3 r = F F és : 
apply. behafforal principles to develop prescriptive techniques: are doing. 


€ 


ig -wrong, or not useful. Data exist quite to the Contrary. The point 
: es 2 : * ’ 


- ; 1Ox®) 
, ‘ : mm * _* ad 


T 
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* this regard, Wells Hively -has made somewhat the same point that was. ° 


| 75 
ie ‘that when it ‘comes to os ceeiene me iedatone, aie use of datiy Qr 
. a 
éabbine “repeatatl behavior seal ae and visual, display of data, igi 
; make a unique |contributiéon. | 0 
‘ ; ' - # , 4 . - 4 
Since che pokeatta, of “time series data {n assessment ig our primary ° 


- 
. 


interest, the Lnpoteant distinction for us in tasting is not: t-whyther 


x 
\ 


assessment is norn-referericed or eriterion-referénvgdi sath it is 


e 
rp 


whether assessment involves single or repeated behavior, sampling. n 
es ' eg ee 
or ; 


9% 


made ‘earlier today -- criterion-referenced,’ tests and norm-referenced 


by 
me x » * 1 of 
tests can be. used for -the sahe purposes. Orfe can. convert most norm- » 
Pa 7 \ Bi 1 , . “ 
r - : ° ‘. > 


refefenced tests into criterion-referenced, pst, and’, vice-versa, 


. 


,« 
4 


4 


e , Yepeated measurement and 


. Wé take then, from pehaytord apc nclos 


"time aceies data analyetas it is. around tha 


a . F 
a to assessment ae 


. 


» while.a review: and synthesis 


/”" 


we have developed a program. ie research Ka 


a i y 7 
° 


off the area. wuggested the, pogential contributions of behavioral research 


4 4 


methodology to AdBnétfLeatiton and planning decjsions, we have restricted 


e * - o 


’ 
t ‘ a ry ’ 


ment for purposes of making formative evaluation ro a the daily 


yu rey? e - . , a 
ide teins made by teachers in ‘an erent to amprove a progr am which ts: 
mF e . in! -5 


in progtess. ‘Thus, the primary focus’ is on eden donee th can be accom- 
pfished by the teachex when instructing,’ or managing, aie, 
e "a +f ‘ “ ® ae 7 a 
‘and which can ber ued Ho tailor the in#truct Lona program to the 
™ , aa 
dndtvtdual stndgit “in an e&fort to make *hat instructional program .~ 


. ad ‘ ‘ q . 
more successful, At this time, the researgil does not include jeing a 


rf ry 
behavioral approach to identify chijdren with learning tteabilities, or 
he : 


to prescripttvely ein Proprans (eteheden We wild be involved tu program 


4 


4 
CD 
an 
ws 
eo 


the gocus in our research’ lane The decision made was to focus on assess- 
t * . .Y . 7 a* 


noe 
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, Planning). We expect, Rowever, that the research wili have implications. " 


x 
i 


for all phases’ of programming. We simply ‘will not. work on all fronts 


_ simultaneously. ,At some. later time, perhaps, we will have the opportunity \ 
to try to do that. : 3 oe * <. 
t ; vos 2 
: . 4 A . “ ~ 
mo, a The Research Proposal - ‘\ 7 . ; 
7 E ¢ 4 = bad me 


To this point we have considered general perspect{vessywhtch have . 


: : : t 

provided the background for our researche We turn -now, more specifically, 
° : : % ae . 4 

to the research, beginning with the assumptions upon which it rests. a 


- Assumptions ; | me a = a ‘ a : 

ae | 2 a 
The first assumption, which I believe very strongly, ds: ° 

At the present time we are ynable to prescribe 

“Specific changes in the grams of individual 3 

students that will certainty be effective; there- a 

fore, changes in a, student ' s instructional program. bs 

should be treated as hypotheses, which must be 

ey eae tested; 


If you do group peseagen, you cau identify interventions hich | 
a y | 7 ‘ 
amight work in genera] or might work in a high proportion of cases. | Lo 


Spectql education, naar is. required to do individual asseedment ‘and ¢ 


Programming. That creates an -incredible technical prob1¥n. To sieatet 


‘ 
e : Y 


: that a particular intervention, even a placement in a resource "eoneah 
\. in ja special wieeos will be an appropridte change Fie given child 
ne as think, to requiring ee Safety Council to pre- 
dict not only how many accidents there will be on the fourth of July, 
oon who will have those accidents. [I could be wrong, but J think that 
our ptoblems in seceduannt and programming’ are eeeiavous to that. In 
a ca OM 


.individual fases, it is very difficult to predict with any certainty 


* 7 ! 


o i. 8 
whether ahy kind of reform that we make in the student's program ts 


‘ 


* 


* 


going to benefit that individual student. If you believe that people .; 


cal 


fea \ 7 3 a me 4 
ad 5 : = ‘ Sy 


fo are unique ind{viduals interacting uniquely with theJr current environ- 
® m a ; . a 


“ment as a function of their unique training histories, and their unique 
constitutional charactévistics, — ~ difficulties in attempt ing to m : 
make a‘ reliable prediction about hae program, . what eee what _ 

: ” 5 ~“~ = 
incentives, managed Dy Gide Hedchoe, AbyiousTy exceed our pechuiest com—_ 
D \ 


“petence. A very highly structured program we for a lot of 


. 


’ of : 
i 


children’: but, to predict exactly which kids that pr gral going to work. 


ot ig virtually impossible. Phere is always’ that residue of atudents 


a” st a 


who do not succeed” very well tn even our best programs. Further, those— * 


kids who are ‘ eucceeding May be apabie of ‘gre success given other 
: i ‘ * 

program arrangements. Our view, 

' t : ; 

is best for the student on an a p¥iorbasis: you must continually monitor 


the’ performance to ‘ensure’ that what you are doing is benefiting the child. 


you cannot be sure about what 


' 


That’ s the ane systen. 


Our second xeudioelon iq that: 
- aac " Special education is an ‘intervention system créated* 

, \ to produce reforms,in the educational programs of 

Tp . sele¢ted individuals. , 
Wone of my favorite articles is one by Campbell (1969) appearing fn 
the” American Psychologist, entitled "Reforms as Experiments." His 
. ¢ , 
point there, a8—mine has been, was that we should treat all Gominrstracive 
° v ° J 

and educational reforms as ‘experiments, doing the best possitile experimental 


analysis of the epfeute of those reforms that we can. , Special education 


\* for each child fs‘an intervention or’a reform which can, and now‘with due 
x ; ; 
process requirements, must be ampirically tested. 
A 


From our primary assumptions it follows that: ‘ * 


The effectiveness of learning disabtlity services 
provided for an individual student will be significantly 


. ‘ a 


determined by the capacity of ,those services for 
cont inuougly assessing | the effects of program 
adjustments which have been veotence to i 


program effectiveness. 


This assumption is critical “to the propqsed research. (fhe assumption 
=e 
; 4) 
implies, in a statistical \Yense, thé” of the total variation in student 


achievement, a stgnificant proportion is going to be accounted for hy 


“the daily instructignal d cMions made by the teacher. The assumption 


is that what the teacher does in making adjustments in a student's program ~ 
7 7 . ~ : 
has a significant impact on the success of the. program. 


2 A final assumption is that: ‘ és 


. 


Teachers will have to be capable of implementing and : 
manag ing ‘systematic formative evaluation whigh makes , 

a program responsive to ae effects with individual 

students. : . ; - 


- If it is true. iihare the effect of baachoe decision oak ing is sig- 


nificant, then teachers must be Abie to continually assess ‘phe effectiveness ; 


of Gat thes are doing. This implies that if we can co tote better formative 
evaluation systems sidyet Peckivelly teach tadchers to use those BY StCME: 

they are’ going to be more successful. [ believe that the really important 
things that happen to students ar@ what raachaes do. from day to day, ata 
that we can seatg aed influence what they do by improving the asseasment 
apapeduess--beathars use in making these routine decisions about reforming — 
4 etddentts program. mo a. : ~ 4 3 


‘ : Y ; a 
I believe that systematic’ formative evaluation systems will have to 


be identified or devaloped which can test the effectiveness of adjustments 
{fh a wide variety of program variables. Consider first, changes in level 
of service ag a variable. Our evaluation systems should help us to deter- — 


mine whether or not changes on some cont'nuum of services from Level I to 


“ 
4 


oe. " 
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Level II, to Level III, and so on, are benefiting the child. The evalua- 


tion system enous? also assess effects of changes within a level: Be a 


a student is receiving thirty miiuges oF Lp resource a a and the 


1 


time is then increased to pceyeiPive rs or an hour. It}is desirable 


program réform occurring -within 


o Ns qe’ 
one level of service. The formative evaluation systems should allow teachers 


to be able to appraise the value of th 


nd Other decision makers to determine whether or not such an increase in 


amount of service has an effect. Changing the amount of time in instruction 
is important since it involves additional resources. At the same time many 
believe that,amount Of -instructional time'is a significant variable in 


determining instructional success. 


. 


j Our formattve evaluation system should also allow determination of . 


whether or not, the changes between curricula, such as those between dif- 
Last - pas . 


ferent reading systems, are significant. Further, since changes occur 


ry 


within a curriculum, such as skipping steps or modifying the skill sequence, 


it is important to attempt to objectively assess whether or not changes 


o 


soot e 


within a curriculum are benefiting the child. 


A chad: set of variables to consider is changes in the type of 


instruction or _intepvention, “Introducing an sine Lone ingency mManage- 
Ment system, ‘providing oral aeons practice, or providing auditory dis- © 


t 


crimination training, ad a at program Felons: which may’ or, may 
N- . 


"1 


yo 


* 
‘not be benefiting the child. hg with changes in level of aT and in 


curriculum, the aunber of changes, ii inétruction or intervention which 
od 
can be created is almost endl 55, and tele effects are unpeed ictable. 
’ 
To compound thegpyoblem, even when a proyram appears to be working,-,it 


Y 


a? i . 


‘may be necessary to change anyway (cf. White & Liberty, 1979). 


ae . ‘. ‘ ~ 
i - % 
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‘Gonclusions . gh a wg » _ 
The assumptions specified .above and the perspectives from the - “ 


- behavioral approach to assessment can be combined into a set of cohclusions 
ow : 7 ; ts . ‘ : = 


about research which must be done. 


e 
7 


’ First, time: series research designs, based on repeated measurements © 


. oo 


~ of student performance, which behavior analysts have developed, could 
be used to systematically test program adjustments. It ought to be | 


possible to evaluate the effects of changes in learning disability services 


by using the kin@ of data collection, and display systems that have appeared 


7 e 


for years in the Journal of Applied Behavior Analysis. Often, -when I 

a 7 — 
make that ‘suggestion, too many people react by gaying, "Hey, that's behavioral; 
what ‘they are looking at is the effects of reinforcement contingencies on the 


behavior of the students. The children I'm interested in are learning 
digabled. That behavioral’ stuff does not really apply because the child's 


problem is not a reinforcement problem, it's a learning disability problem." 
Sa 


So the research is set asia} without considering its’ potential. I am saying, 
, "Hey, wait a minute, you're\chrowingijput the baby with the-bathsb" Whatever 
your ee biases, there is a really good analytic, system involved, 

one which might benefit ger cuiidves about whom you are concerned. You can use 
“the. behavior analytic research me thodoldgy atenie-eabecc tae either behaviorism - 


or operant conditioning. Individual subject time series system analysis can 


fy 


help you to evaluate, in a formative way, ay effectiveness of what you 


? 
’ 


are doing. ‘ . 


A second conclusion is that, to be effectively used by teachers, the 
) ; . ry : . 
time sertes researc designs will have to be embedded in forma ive evaluation 


procedures which can easily be used within the context of an ongoing 


learning disabilities program. ., One of the major resistances to repeatedly 
‘ "a « ‘ 


‘ 
a be e 


b ‘ . 
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Measuring student performance is that (4s sich an intrusive activity. 
Teachers question whether the time andMenergy neveteices tt ee 
The question pleat asked is, "If I spend all my time testing, when am 
tT ‘going to teach?" We pica. address this concern in our program of 7 


a research. 


A third conclusion directing the research program is that formative 


evaluation systems should include specification of target behaviors which 
a - : eee 


¢ 
¢ 


n the curriculum domains for which learning 


’ validly index improvement 


° 


disabilit 


rovided.' We should create simple and 
: 


direct procedureg fer repeatedly measuring target behaviors, for graphic 


- 


display of repeated measure ent data, and fom using Ene Bg phed data ayes 
_tematically to cumulatively improve a program. i Yaw: peoposine that what we 
7 y should do is develop procedures that are aaittexeus. I sippedés to what ata 
‘“ sictans use when they measure vital signs. We need to know what are the 
vital signs gf growth in reading and other basic skills which “are wes dees . o) 
measure and .can be dead by teachers to monitor the effectiveness of what ‘ 
they are doing. A first part of the restarch program, then, addresses it~ 
delete the identification of target performance which validly indexes im= 
oko venank in the curriculum domains for which learning <eanwiitey services 


are commonly provided. | 


Research Objectives | : an 


Our primary three-year research objective is to empirically determine 


whether teachers who are using ue kindg of formative evaluation systems 


i" f . 
4 ry J 


that | have been describing are more Orreetave in amp EON Ine the basic 

' _—« & 

] academic skills and the social ee of learning disabled children 
y ec r F 


} than poachers who do not employ” such So roaable cums evaluaticn. 


We are trying to achtfeve that ta three stages. - . 


— : : 
\ » ° { 
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. The first of these, Stage 1 in our general research plan, seeks to 
. answer the. following question: Can a simple set of behaviors be identi- ; 


B.S 


é P 
fied in reading, spelling, written discourse, and.social adjustment 
which validly index improvement in those domains? Some other basic skill - 


areas have been left out deliberately, not because they're unimportant, 


but because we have only so much time tp devote to this activity at the 


« i 
present. « 


~ 


* Secondly, if we can identify behaviors as validly indexing improvement 


in those dotains, can simple measurement procedures be developed for those 


behaviors which teachers could use routinely, daily 1f necessary, to monitor 


the effectiveness of interventions designed to change those, behaviors? It 


x 


is one thing, of course, to identify a behavior as a géod a of growth, 
and it is another ésehien to develop simple procedures for routinely qb- 
taining date on that behavior. | 7 

Third, does Savicki in the frequency or’ type of measurement affect 
outcomes? That may peat Like a peculiar Aieeeiens but Tom Lovitt has talked - 


a lot about the value of Utrect and dadly measurement. For purposes of 


a 


developing formative evaluation procedures ‘to evaluate the effects of ~ ty 


’ 
- ry 


1 «4 ' e 
the kinds of changes that we are talking about in currieulum, instruction, - 
and level of’service, we would like to establish the, importance of daily 


measurement. We would like to know whethéy the teacher can drop back to 
t e . » 
less frequent measurement such as three timesB a week, or weekly, or-once 


‘ 7 : 
every two weeks, or once a month. We also seek to discover, whether it | 


as 


i 
makes a difference if. behavior is quantified, as percentage, or rate, and 


‘neces thé duration gf a ee sample “is one minute, three minutes, 
or five minutes. Variables like those, which are parttof peasgurement 
itself, can conceivably tnfluence student performance and teacher decision 
making, so they should be examined empirically. ve 

*) i fos ry . 


ERIC 
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~Qn 


-An additional set of research questions relates to whethey systematic 


techniques can be identified which improve teachers'-use of daily assess-' 


ment data. A few studies have been done in which specific rules havé 
governed teachers' use of.data to decide when a program should] be changed. - 
Ralph Bohannon and Phyllis Mirkin both used such systems in t eir doctoral 


research and obtained positive Sin, See Liberty and Owen White . 
. ‘ ey : a 


are the leaders in this particular area of research. Our question is, 


if you have teachers using ‘a data utilization system, does that maximize 


. 


the potential contribution of time series data to improving a student's 


,") . ; t . : oo i * 3 


program? 


A final Stage 1 question is, does variation fn who does the Measurement - 


- inffuence effective data utilization? Teachers, students, or parents can 


be involved in data utilization. Does it matter who is involved? 
We turn now to Stage 2 of our research. Once ye have answered the 


questions posed in Stage 1, we will attempt to construct formative wealua-. 


> 
tion procedures in each domain which are logistically feasible. 1 thank 
Tom Lovitt for the notion that, ‘we need to consider logistical feasibility 


within the centext of school learning disability programs. To do so, 


we will attempt to determine whether or not the procedures we develop 


are generalizable across different curriculum methods, ay a organi- 


zations, age and grade levels -- whether it is easy to train people to 


use the procedures, whether the procedures are efficient with respect to 


teacher and student time, and/finally whether it 1s easy for teachers, 


’ and‘others involved, to manage such systems. Those are critical vartables | 


ry 


* 


in determining the Likelihood that such systems would be adopted and 
used within the schools. ° Logistical concerns, are major ones for nnawering 


our primary research question. Teachers must be able to use our formative 


j : o 4 . 


| 
of 
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evaluation procedures over the period of a full school year. If the’ 


procedures are too cumbersome, we will have difficulty maintaining teachers’ 


. 


f 


measurement behavior, ° 
If we are successful in Stage 1 and 2, we would like to do a major 
experimental contrast between the effectiveness of teachers’ using the 
formative evaluation systems developed in Stages 1 and 2, and teachers' . \ 
' not using such formative evaluation systems, | 
, 
I will close by providing a, hypothetical example of what we hope 
“to achieve. Consider the graph in Figure 4. Suppose, for example, it 
turned out in Stage 1 that reading common words in {solation from some 
high i eleaency word list; like the Dale Hist of 769 easy wordef“Is a 
valid index of the’ student's proficiency in reading. Suppose further 


‘sy | 
that we could generate a display of the student * s performance using aA 


one student's performance from dnitial seghement tbrough a ueelee of / 
program ponenns 7 numerals on the graph represent reading £ from tye " 
isolated oul list doete each program phase -- the, rate correct and 

. incorrect, ‘median rates, for ehose phases. The first reform (placement ® 

" deeiston) was to have a learning disability resource teacher congult 
with the teacher about some possfble curriculum change. The changes, te 
sequentially, were consultation with a curriculum change, but continued 
class placement, and then a second sateen change, 


. 


ve Insert Figure 4 about here 


meee we 


“As you can see, these reforms did not prodtice an improvement in 


reading performance (dt least as indexed by our measure), so @hea we 


a- 
4 


“we hope to develop and test in the final stage of our research. 


ae 4 4, oo ss 85 
changed the level of service to resource assistance with one-to-one in- 
struct tof for a period of time during the day. In doing so, we maintained 


at 


the second curriculum change made in the classroom. The data indicate 


~ 


_, that the reform initiated tT in reading, but a bit later the growth 


seemed to level of £. What We did then wad ‘to introduce an instructional 


¢ 


change -- oral re-reading to mastery; again performance accelerated. This 
" . . . 12 
hypothetical example well illustrates, I think, what we are striving to 


achieve in our research. A system like this is’ the kind of. system that ., 


v 


3 
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: ‘ : é . Behavioral .Research Methodology: A+Reaction 0 _ 
N rr os ai - % . ' ae . 7 : a 
ro, o. ri 4 : ‘ i : : . 
is Thomas Lovitt’ -* * ~ ~ ee 
: : University of Washington ,° .* 
, : .¢ ‘ w _ Zs 
« N ‘ e 


t will organize my Comments atound the ‘three features aE the research 
program on which I ‘was asked to mae coments, and ‘then talk about the 


six ‘proposed research objectives. t oiiisaien preserit .some ‘ideas on * 
‘ ° ~ ‘ . a 7 

a N applied weasarch:¢ and where it seems. to be going. 2 — 7 

’ : l will first comment. on the, Rn legos It’ ‘seems to me that, 


eoce 
. 


oR ag, 8 there are’at least three ‘Htpratures related to this area of f research, 


’ 


iL think one would be the Literature on precision” Feaching as set ners 


from applied. behavior analysis. Alth Bhi dividuals in this area cand’ 


. to be" ¢ Live fore ack car 


rv 


R56. and, ften “ poor wr ters, Eney are 


= “A rathey creative bunch, and t y move ather: mscuey from ene claps to 


another ..” One hag, to ne arbit c reful in reading some apes ite tature. 
| ‘ . AD second bedy of relévant ‘Literature comes from the applied betiavion\ an- 


° . \ 


7 . 3 alysis. camp - from metec the Bijous, the Wolfs, the Bernbauers. — 


set/of aauiodolaytas, can* be. gained. from them. I think sometimes thay . are 


‘a bit impractical in some things Gach as’ school systems, political issues) | 


¢ _ 4 bit these researchers tend to be more applied than some people. Certainly, 
. _ @& third body of relevant literature that might'be considered more strongly-* 
\ 


by Deno! s Zroup would be ‘that in eantuat education. - the International’ 


OS becca! Association, the John Guthries, ithe eal Samuels, and so ‘on. They 


ea ) 
“fa have _— working on reading, mathematics, aelence, P Minnesota ‘state ro 


, * tory, and 80 én,* for a long aces oe think that if Deno's sroup., can . pull 


together the: literatures from howe chree) Areas « précision seaitee, Pe 


oN 


"4 | plied behavior aflalysis, and pederal ciinriculam, we will benefit greatly. 
/ ee The gceata aspect of this research, on which I will comment involv 
: - «. 
y oo _, the rationale for the research. I couldn't. agree more heartily with/what 
. L.. group is doing. They are working .on, basic: skills ~ reading,/ 


‘writing, and: arithmetic; they are working in applied settings - schools, 


we’ ot 


_and yehey. are galing, with a population ‘that is weno eae = learning 9 
-_ "disabilities. What, could be | Bare _— rs tse 

‘ ; The third point I wryfhrstves ve relates. to the potential qgntribution | 
; .- ~ we f 


My resounding approval here ‘is based on the = 


of the peat esa resear 


., . same reasons I gave farlier. Deno .and His cohorts are’ studying fundamental 


4, 


eo 


+ 


fild now Ltke to run through che six eo of - research 


e a few comments on them. The first objectave’ is ‘to 
as: . 

nadine: apeitiie, written discourse, and social adjustment so ree ene 

a offectiveness of LD intervention can be monitored. N hope | that cia ieees 


7 


not mean that you want to find’ dependent variables that: are niandy oc eensi- 


easure behaviors 


° 


{ 
tive to some of the -old. stand-by LD interventions Such: as the Frostig, 


walking rails, ‘amphetamines, Distar, individualized cubicles; and DLM : 


materials, which are commonly produced. and ballyhooed. = think that night 


* 


) ‘ 
_ be a backward approach, and I really don't think that is what Deno’ had 4 
{° i 
ahs mind wat ere _ ‘I think they are trying to ideftify important behaviors c 
that we “shoutd deal with when: we | Are. trying to educate? learning disabled ‘ 


So Achildren, In sdenttstying important behaviors, you could just. simply ue 


\ ° 7 
oe £. a teach LD chimes the same behav lors that aré ape to. normal children, 
i as has been ddne in the past. de, you aight identify” the behaviprs of 


4 Lae ata 
successful adults and. try :6 teach ose 2 Jearning disabled children. ‘ 


°. ‘ 


_ Still another’ way woyld be to use a vada corps, an independent, Broup 


“4 of Judges. Such a 8roup would verify the existence of some deficit 
’ a - ; oa a 


ar woe te a . ry oe | . . 


/ ERIC” 


JA Funrtoxt Provided by ERIC 


or Rgnyoapeave Perey tors. eae after someone taught this set of 


behaviors, would tedvaluate the ehilavea and Spee Ol ty: Say, "Yes, how 


s ey 


they | a OK, they are igh-LD. " Genceiuaeiy then, you have dealt wath 


> 


ae Catstioea. : think ad is time te really grapple with bigger ra 


eee t “9: . _ as 


designs, more ai designs, and perhaps go. ‘beyond some’ of che in- *; 
dividual charts that have been used for so many years.” 


Ww 


The’ second objective of the research isto measure those behaviors 


q 
(Feading, Spelling, written, discoutse, social sadjustment) . Deno referred 


bs e 


a és controversy about measur ing "in perms of erenugncy,: duration, per- 


~ ‘ 
« “ iS 


centages, and whether data- should be collected on a daily, weekly, or 


moneniy baates and - - “There is also’ a great deal of controversy on 
_ how. the behavior ‘should be Seanhed = ‘on the Cartesian coordinates a 
Stanley used or on semi logarithmic paper, that Ogden woutd use, or a 
Some ‘othet form of paper, There _ rane de ass on these Bgeueo . There 


a>" 


are several ways to: resolve these nethodological problens, but again, IT. 
think one.way would be to have ‘a. validation corps, a ercuy of judges. - 


ee v si ' 


if 


é 


Of course, anelddtaeicn would’ be cost, such aa the: denath of Eine 
. 8 . ¢ ‘ e 7 
aquired to teach one measurement system as opposed another, = 


om third research objective is to determine who ghould monitor and* 


‘ \ we ¢ 
® 


, Measure the effectiveness of, an intervention. Reliability is cxtrenely 


‘ 
e¢¢ 


. ‘ ‘ 
” -dmpor tant here. Often we.use a friend, BHOUse +: or roommate to be our 


second recorder of the data, if -in fact we use a. aédond® Be Sata We. ~ 


* A 


- ¢should be ‘usin highly ° trained, non-involved, ‘and, wioperusive indivi | 
‘duals to crosg=check sour‘ measurefent. In’ prectices st think that the 


effectiveness of intervention should be -meagured by, an immediately in~ 


volved person, stich te the parent, the Eeachien', or’ even the child. 
The fourth research ‘objective is “ito insure that the. data obtained 


* e . a *° «# 


Ry | e ay le ae , 2 . , \ 


” 


we) oe = l é a ae ‘ ’ : “ee” «eS ‘ 
. in ‘nénitoting an intervention are utilized in improving an intervention. That! 3 
N ’ . 


tough, por the atisver is fapoveane:. 3 if data | are obtained, they should be 
, . . _ used in some vane «. They could be used to communicate. they could be used 
_ _ to. ipetidate. nue they’ could also ‘be oe ta’ make decisions. How das , - 
. ‘ | this ptesents a Peat problem. Some studies is indicated that requiring 
me 


, 


s 


i ate teachers -to collect and Look at bhete. Ga, more often aides not help them " 


make 1 more intelligent. decisions. Often, tt seems’ that specifying decision” : 


1 

v 

° 

‘ i ’ 


; ; rules. on, ‘the data chart is an: effective vay “to encourage decision. making, 
‘sy 


~ ‘ , pate Genézy, at the riveree of Washington, did an Incaresting dissereat. 

i : tion along that line.. He found that when tay talked to. ‘teachers and : 
J | my pxoiipeed. them to make decisions based ae ana gave social reinforce~ ss 
| 7 _ ment for doing it, the kate’ of. making decisions ‘Ltacredsed. - oo 


. The fifth ‘research objective ts” to identify. systematic formative 


. | evalyation Procedures that aré logistically feasible within the context 


of school programs. The first problem - is that no one has really identified ‘ 
» A 


‘ . 
* 


the various svateacte formative evaluation’ procedures, aieheuah Deno — 


4 


did to sone extent. pecond, I don't think ae anyone. has. determined the 
- . extent to hich formative evaluations are now uséd in school programs. te 3 


“Ke Third, I don’ t Know whether anyoné has tested how often and with how | hee 


s 7 many euauee et se systems coudd ‘be used in the ‘achaol setting. If a’ 
eH eG teacher were to intain four or = charts .per day for 30. to 35 students, ps 
<* \ that would be 125 r 150 charts per day. It would be interesting to dust. ee 


1. Pg tS 
test for Limite on tven sets of circultstances. "However, one must “dlso | ae 


consider» the politic and the realities of it all, which heang we have to 
a deal with busing, unions, ‘collective bargaining, tenure, and so on, . 


r 


"€ ‘ = oF = ! 
when developirig a data system of any kind, ~ 4 3 ore 


: - ‘The last ‘research objective is to deternine whether ‘ayatamtic. Mig # y 


- 8 s aa " . 
@ formative evaluation = improves intervention. effectiveness. oe ae 
cy A Sd ' ; L490 - & 8, at Kae 
> 


ro, ; : Ad, o Re omg 
. ag i . ne 


ERG: 3) Sx, 4 


3 = ne “a ° = 


1 oo. Hoe, 


thah nothing, ‘whether - it is mawe ee eee ive. thdn another data system, mg 


oe : ’ 


» 
. = and" so on. You seem to niet to do this.’ * . 2 os Bae 
* * * , @ ‘ é ‘. * 


+ . 
, 


a 7 os I think. nate: Pane research is peering in,a different way. fo 


‘It's not a better way 5 it' 4 a 4 different way. ae think that research’ 
; / .. oe = 
Problens are pee toning to emanate ‘f rom the schools. Previously, we 


7 


’ 


ns migne have: gat in a ‘Library or, in a cubicle and jeduee. up a research 


a | idea, anes after doing. that. eeccaees and coming up ith ‘significance 
. ° w we 
"7. at the .001 level, we would wontler why no one Pup lemented it. Now, 


‘ of . 
' > t ‘ ‘ 
\ 


research is beginning to’ eapbad to ‘the needs of people in .the mn eOR 


a “ICY. . ue QQ. re 
= 7 ae this as-one of the — tae we dre — through ‘in applied (Mee 
. . | iaiagieg analysis. ‘I pink tet another ,change is in the easires that ‘ = 
ara — "ae take. I think we are seeing. more data being gathered. wore: and nore; 
. : ) 
; I think that as as renearehess, we have to Bet. ne, peommneltye Hee : Je ge 
. , . \ 
= : a . ‘People, involved in ‘what we do. We mre talking now, about social signifi- 


me ee as well as statistical signifteances Finally, I think. chat” the 
3 * \ 2 ames Yee) - - ; 


be coe ways in which we | disseminates and validaté our ‘programs are beginning: to : 


’ 
at % ’ 


» 4 + Fechatge. .The important | erdtarion in Seternitniny, the yalue of. Hesearch or a 
the number ot reqhésts for information from teachers and other éonst'- ; 
e wy 6 Re j = » 


_tuentiss: how many are ‘used-rather than just put. on shelves? . Were there. 
= atly effects, and ag 80, of, what magridtude? And, how Many. suggestions 


4 - for: research doe} & the researcher get from teachers or other, constituents?! 


cant z . . om ‘ 4 : 
+ % : ~ : ‘°° es _ : md ; eu 


_ or he has to do is scream . for. someone to help and that, someone will solve 
: bd —, 


and I'm eeene that there will be, pnd enere has a in “the work of - 


| ” ke : , § . en %* — ; , 3 : ' ~ e ait : 
Behavioral Research Methodology: A Reaction . to. on i 


’ . ‘ 


Phyllis Newcomer 


4 . , , Beaver College oy oe tl oe fag 


ay i ‘ cy : cd 


I would Like to give my Ampresstons of How I ‘think this research ; 4 


oe can raed the field of learning disabilities. I also want to.: «' 
- ss ae 
talk, about some of iy concerns, % Pe . ve 


? be 
“find the behayporel re exceedingly at eaneiges ES chink ; 


a 5 3 places the responsibility for, assessm ht, for faatructisial purposes, 
: 


ptectsely where 4t- Relongs, is that! 3 imarily in the hands of the Cees 


- 


eaten. It. gives ‘the teacher amethodplogy for’ sequencing, the’ ‘learning: 


e . o 


activities in which the. child needs a The wrecher 1g ajso. = 


disimbued of the dans vr if a ‘child has a iedratan neva, all she. 


the problem. Therefore, I see, ‘many’ advantages to” the wee eer approach, 


—— 


Sthers, some very euccueietae evidence; that’ applied beHavioral analysis “a : 
‘is effective. . "4 oe “oe A ' 4 
7 * * , | 


‘Now let,me tutn the* stoin around eat tell you about. some of my cons @ 
cerns. I am not “associated with one aneettte approach ‘to. the diagnosis‘ pk 
~ . cake 


? . . 17 


“and remediation of learning disabilities. . However, I have had expan 


with ey: cHildren: ‘who have been called learning disabled, “and 1 have nee . 
yb. 


developed certain impressioxs. You have indicated previously that part ° 


\ 


‘of your belavisral aesdssment approach will be for the purpdse of idé at 


teteatton, i I was gracified when you said that essentially, thac purpo ee. 


ws ° 


, Was not {the focus of your efforts. If ‘you 100k at any of che! definitions 3 


! 


that oa currently prevalent,’ it is very difficult .to fit your behavioral | 


R ‘ a 


model for assessmént, and particulgriy. your basic umption that learning 


disabilities isa natural product of. che interactiaf’ between the- ‘child . "de 


: * 4 4 


oo : a 
fo 4 aoe m 4 . . ae . ry 


"and his as her environment, into those defin tions sinde they preclude 
I, 5 : ° : . . 7 ee ‘a, 
"4, calling children "learning disabled" when they are primarily environ- 
’ ‘ al 


; . - 
‘ , mentally or culturally disadvantaged. ‘a think you really had to deal 


° | with that broblen? in order to. justify your aeoPonch to identification. 


“T am far, more concerned about: the specific plans for demonstrating .. 
a : 


eo" , . ey » 


uw oghat this research ‘into behavioral mei hodeloey is teally going to benefit 


ee the’ group of children whom: we call learning disabled. My interest is not 2 
rn oe , ee: 
= only,’in leardiing, mote about behavtoral methedology, byt ‘in Léarning . , = . 
: a ¥ iu 0 : , 
iq an -" more about téarning a I have some quate about your’ pee 


. 


popubations - children who have been ibetaae according” to inconsistent , 

criteria as LD. I wonder what conclusions you will be able to.draw ree = - 
“3 aveh Cecleacitgs disabled children after’ you. have coupletcd ydbr ‘study. . oe 
oo ne | I can appreciate that, youg will be able to draw many conclusions relevant _ 


f to children who have experienced academic failure. I. recognize the im-, ry 2 
’ o ae ‘ o . * : ) : : 


: a portance DE PNpEE conclusion MA realize ‘that. behaviorists . prefer to °: : 


“ 


" avoid the issue of etiology becalise they regard it as insignificant. wnak 


es is pera is “developing syatematic programming to overcome pear ENe 
deficits. . And I can spprggeate that position. When we oy to operation- 
alize the dafinitiéns of learning disabilities, we’ ‘often want to tear i ty 

er \ naa A ot 


dur hair. out, particularly if we are in an ‘ehh icotment mere we ‘end up 


with 50: percect of our stidents as. Learning: disabled. -I BEree with \ Tom. 


is - Lovitt aa the people who generate: these defiaicione very: rarely ‘use _ Ri 


”. 
° 


an. empirical base for then. However, while IL apprectste “your position, 


« - & 


ZS am concerned, repeeselly. because the’ Institute is funded Specifically for. 
‘ * 

research bares tan: to learning disabilities. - We can “get entranced. with 

eae approach: and still ‘end- up: with Liteie pertaining ‘directly - 


+ to learning disabilities. | : ee gt 


. - a * € 
* on e - “ . r . 
. s : oe . ie : 4 4° * . ¢ lye, . 


A related issue ad 1 m going tolcomient on is: the person-centered 


: vt - 
g lee approach for remediation as opposed to ‘the c sttuatign-centered approach: x 


aan : ; 
* » 


- 
e ' * for remediation, in a’ benacies aa Ji am a firmly in favor of working 
a, with the. youngster in. the 1 natural termes the eiaseceom: -I pave 


* 


7 s felt ehsrewey for a “long ve even when’ the typical approach to ‘avchitd’ 3 


( learning problem was to ‘examine and remediate in isoldtion.. I belicie 
that we often, Zot data that wasn Me always as pertinent to the problem 


, 


* Situation ‘as we could _haye obtained by looking, at components within the : 
4 classroom. Meaoe I finmly perreve dn the tntetactio@Mabpects of wy 
va. . problem in any school situation, I. would hate to ere pis rejéct the notion 


“that a child may have a problem that is not induced by classroom related 


ve 


a: a, : variables. “The ereee would be to ‘shift Gonhasde: ‘and to maintain that Rhere - 


+ 
_ r) 


a yy is no such thing as learning disability unless it occurs ta a elaSsroom 


* 
“sitwation. -It's akin to ee "ve seat happen’ in une area of emotional 


& 
+ 


disturbance. When I seh students “in my, inerouuetory graduate course in 


* 


/ ;  vaxcuptdonatiey,. "What do you think causes emotional disturbance?" they 
bea that bt, is a socially determined variable. People are construed as 


“disturbed because ‘society sets standards and dhey deviate from Enemy In. a 
» 7 ’ vw 
‘t otier words, they give a " sociologiéal perspective: I then ask them whether — 


: *  . emotional disy rbance ar wiaage relative to the social sitiption. When» | - 


- they say "Yes," I ‘ask about sae aca Eventually,” they realize that oe 


s 


- 


not all conditions’ are relative to social situations. ‘Tywould hate to. see 


’ a the pendulum swihg ‘in the field of learning disabilities to imply that oP ae 
‘ “3 4 everything had to do with the: environinent; and nothing with. the child. . ; 
“a - 
I ‘don! t think that's eruee a A ; . . 


’ ‘ 
«> J as ok _ 6 : 4 


¥ also wanted to,- note, that\t! m very silnasea that the focus of this aS 
. "investigation ae ae skills... As Tom Lovitt said, what could be 


na i] 


4 


better than study ing. reading, writing, ana ‘so on. ‘However, os noted that, 


be 1 think for: a slight period of time, the new idea period, 


left, the’ new’ procédures were dropped, Sani, bedauwe the teachers 
. 1 


ag: ; 
you propose . toMo some’ of the ork with preschoolers. I am particularly 


® . Ney ae s 


interested An that population: I-share Dr. ‘Reynolds’ concern that we 


1 7 


cannot always be dealing with children‘ after they become sulle tows cas-— 


» 


ualties in third or - fougth aeade? Ll” am. are Lriterested in Fhe kinds ° 
of data that would tell us something about these youngsters before enay co 


start “fafling and develop problems ied basic. skills. I wonder How ex- 


tensive. the empicvtten will be at- the preschool hevels ‘and whether or : 
" * ro ow 
not it will involve any investigations into stat Language. , os = 


Finally, * want ‘to" make. one more eee that réflects iy interactions: 


with teacheps’ It is a: recommendation really. « You are progdsing a very - 
elaborate repaaeild ‘1 khow that,-you appreciate the importance of it-’ 
volving oe teachers thenisélves in planning what is beasible ‘in terms of | -* 


ey, bad ar 10 e ' 


the tiplenentation of. this methodology. I frequently havasteds in aiedae’ - ¢ 


tions where these approachés have been implemented and teachers have .. - 9 y 
. a 
ee ‘ 


hey euseyed ib and everything was fine. But. then, when the researcher 5 7 


s 


did not accept them as part of their réle.° One of the strengths of this” 


‘ 


study could be that. the procedures established continu to be implenentad - 


‘ e 


when the’ research" is ended; | a , . ee 


Re et . yet : a. 


. Behavioral Research Methadology: A Reaction = % a - 
| o ‘ ‘< ae C : oe a. ae 
ee ee ia Joseph Jenkins oy a - 
: ar oe _ University of Hashtneton ar al 


. ry ld cw. . y 7 = + ry ° 
‘Data-babed: program modification. the applied behavior analysis. 
= f = 
- a perspective, ‘is probably effective by "definition. It is a ae E-ebtveccing ge 


2 


. approach which, when done propeily, requires teacterg to "change instruc- a 
tional variables until the behavior being | aasniees is affgeted. In such 


a ae lass pike i thood: of short and Long duration eevee exists since mee 
ae 


: teachers gannot tac with an ineffective intervention. That, it seems 


2: . 


‘to me, is the essence of the ehavigral appeoach on watch you intend to 
. ‘ rm aeoee i __ 


. capitalize in your researéh. "With. that: preliminary comment, let she address 


; r\ 
myself pameeaiere your first ° ‘research adjective, which is\to identify 
se e 


é . -Nsiople. "thermometers" for measuring proficiency in coals, ‘writing, and es 
hd ' a: 3 ae 
> speltiag: on — "4 4 ‘ a i 


= While the ‘objective may be quite clear, the-Lseues that ae raises, a ‘ 


Pay are re cong Thé first issue Ais whether the measures really wil help 
; ’ ; very much. ten! t it possible, for example, that if we made routine 


aN 
BP oe nges without neasYranest, we would obtain the same’ results? . A. second 


Me, ae issue concerns the duecceawel’ of” oe measures you might develop. Direct- tee ay 
; a oe 4 A LG 
" ness inithis context refers to nether: or not a test ‘measurés . precisely : 
shat is being, taught. Suppose. Gener behaviors are changing, behaviors 
” that” may’ ba mote “Amportant, than what your prodedures fueasure? Con- 


e 


ges : versely, if ‘you take rect néasuree“f the skills you 4 are teaching, _ 


as : * 


dish es based on the child's progress through 
your skill sequence, BA are adouming ‘that the skill sequence (or pat 


 riculum) ! is evenly ani, regularly progressing. in difficulty. Only when 


a e4 ws 
‘ * 1 
: » . . 


. . : Se a t ie : 4 a . , % a ‘a 
r : vee x at . ‘ - - Sa eu 
F é *, . $3.45 wet cee a . C 3 . an 
. e . ? é Sk ae r Tat ‘ {- v 8 : 
oa : : ) Pee : e 
Q : 7 “ ‘ : ‘ / gn ae " : “ ; * , 
~ ‘¢ . ‘ ‘ 
: z E i , : so ; : 
“ i te . ee a. : ‘ tos 
] 4 me , : « . _ ' 8 . ; : ‘ J 
- 4 ; , 
. ° 


curriculum segments are, of <r erates: can you conclude: thet. 


a: 
ey 
. 
a. 
t 
“ 
we 


we ; 6 


48 


J “rape Mian the ite eae euetieutey ‘of rests within the je arvieyton. 


‘et 
tes 


~ Re So 4 | 
” nese types of curriculum: ‘packages. changeron aay ‘to 0 aay,” “Ane option ae 


+a piverr curriculum do" not matt well. 


or 
8 


ee we Lae ’ , -. 
“ 


-e a%. 7 7 . 
‘of course; “ts, whether measures. yield ordinal or. “interval daca. , 


ra s 


"the tssuey 


TE you develop ficasires ba ed on’ Pro nese" through the sare Ce the , 
“ ary 


‘ Lf 
ay a eee * ae 


equal interval - equal qfettoutey adeunp gion probably cannot, of made.” aR oe 


vod 


= e e 
Another ‘problem r 


“ e 


“that caching coi 


wry. 4, 
deveropine measures ts, a “seers, to- me, 


measures 


ae s 4 


8 Ps bisear defy the kinds %E 


Tee ornate cm « Tietedee et ee , ux 
Jar-capgoe! e ity nake'sperfo charts @htch re-" - © he teg 


yeu. are proposing. 


° a 
of vy, te 7 € wy. & 


eagene proficiency on: vs yaoke: because’ ‘the behaviors tacight within 3 
v. “4 : 

A ¢ 
‘ts: tog getaway fron direct ieastitement aaa Ro some sort ‘of indirect, 

Lg e . 

pier: which can be: ‘given baie régularly han standardizgdsachieyément’ 
f ’ “é o 
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7 “y ‘-e 4 F - 

“external measure.., Yet, there are probiems ‘wheds euch aie measures and s 
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a, suauideeat: to be a functional jianas being, 3 You can* change almost any.” | ae 
7 - er 7 “specific pehavace ana teach. valmost -gny: ‘kind | of c pibtnce Lei eaeus rw as 
: : lot of high povered talent .on it. But, wha comes. next? 7 ; ae a 
‘e | es generally agtecd. that tha procedure of isisating a varget be~, * 


a. 


havior and t spply tng massive ingervention strategies to teach that. target 


. 
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to fail. However,’ the research is concerned wath contributing to the ; _ 
‘ v. ’ : ee 


development” of a _ functioning individual, | The first step iaree ’ J 


. . “identify indices that ‘are ecologically vatid. Curriculom siete s | t. 


é a ke . 
a 3 ) Were chosen, Eor study Firet because they relate to "dutical {npera- —_ ae 

= a ie ae pa? . 
bn ‘ tives? . ‘ ( » ‘ : aoe o- ae ‘ ateal 
a ‘ ' 2 os : : 2 UF . . 

A . a . ™ . ” a 

- the” research, will attempt. Taeneity cae that,’ when chainged; a8 
rs i 


. Lf reflect. a relevant change in hum cowth aie development. tt wae further 
09". abe 


suggested that an “important aspect of. stherresearch should be. to look at -. * -* 4 


Ler Boos xs ae 
ae changes that occur in other criteria, father than just in the behaviors = So 
ey Ni ae a F 3 ’ on 7 
a ey under Study. _ - ne ae - | . ee 


: : Discusion then turned to thahftinetion of applied behavior “dnatysts 
ay | 4 wT 
" asa éésearch tool versus applied behavior analysis as a. teacher's tool. ia 


se 


o . y 
Behavior sidyads appears to be a powerful_research tool, far di dcovering 


instructional principles that aré effective. But, Vs it a | useful tool sO 


. for the teacher? It appears to be. ‘cumbersome to use in, he AT SERNOOMs 


its ote 
‘ 
a- . : 4 ~ 

; 


Teachéra want things like more personnel aud ae eer materials, ; “ 


but they do not seem: to express any need EOF more data’ to make day tonday 
td “6 . 
decisions on how ‘to change pnbenueedouss What evidence is there to. indi-.: 


a 


‘ cate, that we need to use ‘daily tieasuxes in classrooms? | au Fiak ‘ 
r One type sPeavidenee sentioned was from research that contrasted . < 
. — 4 oa 


teachers taking data with teachers not taking data. Thé research showed. 


the augmented ettecté/produced by data taking. te suggested that such 
. \¢@ \ . . si 
F evidence ig not gufficient. A more appropiate procedure wotld be to 


a ad the elisccs for -teachers collecting data versus Fane getting . 2 ae 
. something elsé, such. .as more instructional materials. . #, ae 
ANOEnES sa ‘was. that the law now poqueree educators” to show 


whether inetruction-has been- effective for a child. There ig an account 


ability basis for cdliecting data in thé Lassroom, ‘Tt was A aygued: tliat 


' . ees a 


7 dae ve Se 


a baa : = 7 
t, : eS expressed re ‘while the “law requires only jain de document sabeeue: 
I: : ? ‘ tages 
. es r . Vv 
= er resgarchers should tabe advantage of. the requirement to determine whether 


- eee can age be uséd tq design instruction. Perhaps, by requiring | 

: | : dois cer lectiaois can pects some. of the instructional, failures that | re 
ae : 4 i an common, a schools: It might be hypothesized that, teac rs wily be | bey 
7" ee — if they are more precise in. tiggiservation make and . a . 


rs 
! ‘ ¢ 
a é 


the data they ep rtece: whether they think Eney need more measurenstit ‘or 


™ 
me not. ah > : 
° _It was 1 suggested that an, n, interesting ‘ine of ‘investigation might he. 


° 


* . to inet at in efYct of data collection for teachers who, . on their yay’ 


do not seem to-have good internal sae acl of what ‘ts going on in 


terms of” a particular child's. progress. Also, ae would be interesting to» ne 7 


‘ e 


.- 
. “. ar 
‘i oe 


eo 
, compare, at some time in the future, the effectiveness of a ayetematic data — 4 7 
collection procedure against an infornal! evenation procedure that is used’ 


| ie 4 in a ccessful instructional program, ‘gush as that used in the DISTAR 

— ‘reading prowan, It te not clear that, syatengtic data. gollection will . -. 
fare ‘so well viven patted against a powerful instructional progrdm. -Such’ ° | 

a muy ‘ipuld mi two" groups using the” snetruct tenet on with: one 


"¢ 


stoup using the beraative date colléetion in addition. to the Instructional , 7 


. 4 


‘ ‘Program. It was : noted that the present, research: does have one ay that | 
“rgpresents, a | firde step in ‘that encore given teachers presenting the 


’ re ‘ 
a . game subject’ nace and uasng the same oceeveationn: a no-measurement group, Yy ‘ 
oe ee =. es 

ye neasureicht group, and ¢ a “measurement decision group will be compared. ae 
\ a ; i is ry 7 
wpa a eee oo 4 Fi 7 7 , 


“Mt 


The point + was made that the. Institute. tg focusing, of. thé ‘methods’ 
{ 


—™, 
apr sprints for children wach learning disabilities, not on t 


of applied behavioral ety ee 
; : 
ue wa ‘nop edmatite a. researchers are ‘trying 


to ) detine whether a measuring 8 agen 8° ‘useful in. terms of the way ences. 
ed 


e 


in! ‘education wat commended ; 


’ 


. . ra 
iz eee i ® 
, a . vs , a 
* os ; . ; soy 
. if et tm 
. ‘ meg) 
4 ne) 
‘ oP 
= * 
e ’ 
: i 4a 
. é : 5 
\ “ - : 
® 7 on 
; ‘ ; “ 
> a ¥ 
; » Kate re 
° 
Fait eal 
a@ 3 
- . & 
2 ° . e 
» * : 
7 ; : 
ys , , 
‘ ker ) 
oy 
= a) 
. . t. a 
« i : 
: . 
. > 
\ 
° 
ry é . al . 
= * . 
: ? . ¢ 
. : ° 
’ 
- “9 . se 
. ee 
a . 
ty : 
\. 
. : . t. 
a4 . i 4 
3 ‘ ' . 
% ft * . 
+ ue . 
, 
op ‘ 
E. 
= : ~ f , 
. ‘ 7 +” 
7 . . < 4 
~ . ¥ ° 
. . 3 cS » . 
4 : Re) « . 
‘ = . 7 7 ° 
* ‘ 
’ » <4 . . . . 
© : ’ . 
. , : & ‘ 
a , as “ 


ad 
« 
4 
3 
Pa 


be . = si 
J chy? OF. 
oe ao 


. . 6 
“ + * t 
nay . . ia ‘ 
e 
if 
“ i 
an ae ° 
e 
2 oe 
R } 
+. 
< 
' 
1 * o Ce 


he investigation 
’ ? _ 


The orientation of the research ‘to success - o. 


Ce 


ae . iy “yt * ccd aS . 7 . a < Z oe ri ar % 
ott — a, ¢ if ~«* Highlights of the Open Discussion | * oo es 
; ns = ° Following Deno Presentation Ms a, 
ety . . ¥. - a a ‘ ‘ 
2 . ae . ee 4 : y ~ , ( , : 
34. & 7 ae ce e.g os oo 


‘ ~ . a ‘ . as 
7 Ve, . ee Biscussion began with gorments 0 on che importance ‘of conducting.” 


tke ; ; - . : - : 
mr. } : “research, before attempting ‘social eartiaeien ofthe technology. It - . 


a is 
2 : was noteld that pftectines technology precedes consumer demand, sae 


_ > 2 t ee mintatupiaed transistor circuits weté available ‘long, before “— 


; 2 . 
«a ~ " 


ee a there was a big danaiia for desk. caleulators, The. pronase, oaeeaet 


is dasiveas to. determine whether an. ‘available technology works for a’. 
~ 8 


given population, - ee requires that wé: have -some ‘answers. 


6 AM While’ some aspects of behavioral principles have adil. valigated, the use ~ 
ae : . 4 
* das 
"y of this measurenent Roche hosy, for trereeatne teacher decision naling a 
3 — , has not been validated. The first hance should be to experinentatty ” os aoe 


‘¢ mote = "validate. that function, before oa attempt is, made to socially valggte 


» | wit. Concern was. expressed that téachers should still be involved§'in 
— : 


‘the experimental validation process. : o 4 


. > 
” 


The aieerien ‘wast posed of ‘vho will determine “what target ie : 


tiaviors to wore oe Is successful learning nerely tiiatever ene” ake /- 
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a ¢ J . ‘22 a _ "9 ae . Se! oe a aes . " : 
eee validation,. in which ‘the teaches ‘Cold, ay whetifier the sige ton oF ee “ 
Ym | So the system was s helpEub, and: whether they “would bontinue , ta use it: if, ~ ar 
a ; v , . as F Lam _ ae . . - . 2: , lg e 
a the, choice was left to. then, Sg . gevis Ste Das a oe ” 
ee . Another ante: ‘of the audierice acueel with the point ‘that’ it  ° a 
1 =." . ol a Ps 7 3 


mish be possible re the significant effects’ of daily measurement ; 


ate ‘related to ‘the fact that the measurement system gets the teacher 
: a to - attend constantly to the. program, There are probably many non- + 


specifig things: that can be dace to focus the teacher’ 8 attention on 


a; ae ° 
i : ae \ } ‘ 
hee : “ ° t : : 


J the students. However, datiy measures do have _ the advantage of being oo 


a 


nleroscopic and thus perhaps not: so subject to halo effects and memery 


‘ ° ’ 


problems: ‘fet, to get ‘the teacher involved jn a way that constant ly 


points the teacher to the: task may be quite a different sort of effect. 


that has nothing to. do with the problem of datiy measures, ae 


eo ° 
It was then noted that perhaps. the Bias single variable by which 


the teacher’ 8 behavior ghould ta control léd is* stident performance. 
j While other ‘methods may inerease the. teacher' 8 abieations it seems 


te 


feadsonable to try to develop a measurement system that oe the. teacher’ 3° 


instructional behavior under the control of the student 8 behavior, - 


so that as the student is improving or not improving, the teacher wild 


make changes. e a , . : < oe 
i The discussion concluded with cpmments on the relation between . a 
7 is ; ¢ . . — ov ‘ 


teaching and assessment. It was noted that the teséarch project views 


e . ‘ 


teaching’ « aa an’ asaésanent ‘technology. as ve’ 48 an instructional beans 


. 


nology.. It is important: to feed data obtained through instruction 


.. Into the decision-making process. Teachers probably are the people at 


oo? - ‘ 
. * 
» s é : ‘ . c v 


aan who ought "to make those assessments for the most part, and feed in-~ |. oe 


aan 


formation from those assesements back into thejdecision-making system. 


Gitmo °° 


aan) ae 
4 ° . 
ee es 
; me aS ed ne i te 1 ; : / * a "a , : . & - ' . y Bs 7 . . oe o F oo 
fy OE yt ae . eum ane POmmneeEy : 4 . on : vee 
ot 7 > ‘: . : a at ’ eee ‘, oF ae : , . as * : * Sie 
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a8 oe cr, c~ingernat ional Readfng - Association ” ~ 2 ee oe FP ee PY gee 


. 7 . : ; a ae ” Sox a . ; e . oy ray 7 Z : ; e 3 
a ts ow _ ¥e 

i? Nigshgeee dldcissioy 8; “and radétions.tn” "this, conférence have te ge me SRE 

ee” ‘. oF eee i ie oe 

| “vevipeed a ‘great’ deal of uiterature offassesament ofcentLareh with 

eae digabilities.. A basic purpose “of bias conterence we oa to ors 

| “state the: Ampoytant problema in the field’ that require research. and pro~ cad 


iY 
oF ® 


pose Anvestigatitons of these. problem areas. 


, 
of r ry o 


My commentary is centered ‘around two ques tone: (Q) Are the e probjens 


> ° that have been stated well-formed? and (2). wilh. the solu to these = #4 
ae | “problems benefit: the education. of ‘children with learning d disabilities? pe i; 7: 


Lt is interesting that these’ two. questions fave deceived more and more .. 


. a ‘ y wot a ee cy 


attention lately tn the research establishtwent. Gaining consensus in the “ * 
> ; a : fm v : Fi a raed 
: research community about what the. amportant problems are has been-more ‘com - j 
: . o . Pe : “abe. < 

fe” mon in the National Institute of Education. ro the last: few years. For exam- 


- q 3 if i . Ay , 


‘ple, ben task age were. ‘formed to develop the. PEPOR CAME researeh issues" 
7 : Vo ; 


An, reading, the outcome of which has. been the Center for the Study. of Reading 


pe 


° - 


at Ene University: Of I1Linots that is focusing on comprehension of children : - 


a 


‘at che intermediate grade level. » The second question, about educational. 


no benefit, has received more attention as "federally funded | réseareh has be~ . 


_come increasing}y mission oriented - Regardless of whether this orientation 

7 - ts, healthy, te raises. stakes for atudies that <4 to promme. immediate. 
- - jupeovenents ne education. a . ; . . * . re 

the . premcuotl for Assessment : 


I have pieced together a somewhat shaky organizational framework to 


1 we 


help me aiduestand the relationships. between the twq main papers, sections. : o. , 
Ag ba a : i) ~~ ba , : 


. _ within the papers, and the reactions of the critics. - It, happens that this : oe 


ur : : u 


e % 
; e . 
a, s \ 


. ns . : 
. + . a ae r 5S . Blin 
f te tay <4 r ; ' 
‘ : Be ee ; : Wot May te ae a! i 
os P e “ : * . ae oe ie 7 he . qa 


. . ‘ ee mo e. : * - 7 ae . . aa ie a hy oof : 2 a ; P ‘ . ie oe 
a = a 7 yy eee ney fc rs 
a we ERemeWOrK, alse Po missing oe that. are crueial =rom at least 


bi < 

) 
% : ae 
-one viewpoint, -thoug 


ey. ‘are’ superfluous from at Teast. one other ~*", 
- ' oe hs wee eo is . Ad oe bd aX . , { 
ae _ vipupoint.: - a = re a : : a. 
oe a rs i Bases . ; . 
Me ee ~ Khe, framevork consists of four ‘cdmponents of the process: of attempt— - 


° - a 
m, 


‘ to: snnEeNe education for. learning: ‘disabledtehtidren. These component) 


. _ ara? wm referral; () ndaesenent; (3) diagnosis; and (4). teaching. a 
“Refer¥al pertéine to the event in which soite person, dbus yeah adult 

sucha as ‘4 teacher, ‘parent, oF principal, ‘suggests that a certais child oy 
or a ‘in’ a classroom: or ‘school ao help usually wine a 
me spate ‘3 fai 1uP to learn a basic ski11 in a’ manner that: makes the child : . o: 4 


e 


t- 


om % 7 2 “unnangigent e by the tdachey or -the ch41d' 8 Behavior that aiarup es learning 
for the ¢hildren.*' Assessment refers to the. administration of tests that: 


y “s are intended to dorciine the child's achievénent, aptitude,’ or social 
. % oo» 


< i: , and attention behaviors in comparison to other ‘children, or to the demands ., 


a 


‘of the cureieutun in which the hitd has. been learning or to the requirepente a. 


yf & the social. context in which ication is taking plack. |, Diagnosis refers. ; 


+¢ a : le 
‘ 


ato Statements. about the* salle that® delineate the problem or to a classifi- 
= 4° catian of “the child into a category’ ‘Based on a Meniliar cdetering of 


-; 
scores on the assessment ‘tests. Andy besaW ing rafevi to different educa 4 . 
em, ; ‘ &, : ‘ 


. , tional activities, iad da ayaCEMe or interventions by specialized 


“ * 
s 


‘ personnel, such as tutorg or counselors,” cl are intended to solve the 


‘problem as: described in the diagnosis. It fe not hard to see that . decision 


‘6 ‘i 


e ° 


° ; a “making may occur when each component or * the Linkage points between 
a ehe ‘compOnenta. be Ss e 5 Wd —_ 2 i. oe 4 oe 


Pa ue Validity, of Components 


. 
5 7 .. 
- Ag * é 


To > determine the adequacy. ‘of a component, or the dacteton-malcing 4 { 


‘ a oe wees y . oar) ‘ . at ‘ 
site ey ate pe i? . ao. e. 
/ pkgs ae: ae * eee t * as 
: * : 


oe * i . ’ * 2 . ‘ “ on 
; . ’ Pa meety) . : ot “ cost : 
a | . ns ee : “a “s So é : a y 
: pe : wee ; : Pa oo | a ca . . a a Te 3 
; * ~~ 3% soe . ance « ; * * et . 
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-Rasenttally, we ead to know how’ ‘to validate each component. ‘one reason 


s - 


* “able ertterian, ehat can be used in the validation of thesd\pomponents: is: 


news each stage: as valid if ie ‘leads to valid gectedops ‘th Lo next stage. 


For example, a diagaosts is’ valid if it leads to a teaching program that 
p 
improvgh the child's achievenent “La basic skills by some appropriate stan- 


.@ 


dard. In turn, an assessment procedure is. valid if it provides sutfictent 


o- < 


information ‘to enable a reliable diagnosts to be ade In turn, a. referral 


\ ts valid if it leads to new assessment" fh formation that is important for 


ry 


‘i the child's education or the confirmation of a suspicion’ regarding.a 


‘cltild's learning problem. » - 


Tecan easily.be eee that this systey hinges on the. validation of . 


teathing. The dependent variable for decision-making research is the : 


4 
validity of teachtna, For our peer Cree teaching may be considered valid 
nen, a enild learns rapidly’ and. behaved ee according | to some 


anced upon sta 


ey as the, educational Progray in which tha child is. » 


located. Wit n. this system valid teaching assumes, at least at some 


. ¢ ‘ , - 


a auchiag system that is. ‘effective | for a 1 defined: aihie 
population” fl, but less ervect tye: for a cts W2.. “At the same - h - 


time, a ditterent treatment or teaching system is effective for eybropula= 


= 


wat 


tion #2, but .less effective for subpopulation’ #1. If this. 1s Tot. the case ; 


and teaching ‘cannot be validated’ in ‘this sense, ‘then the macs ‘of the. 


other components of the decision. chain cannot be determined. "7 ae ee de 


_ The authors at major papers and the discussants in this ‘conference 


have taiten Mocs positions about the cetatiotnipe among Pe 


¢ 


. 2 
- in this Program of research: ig) decision making vinin ah context’ of sau! 


, 


car toning programing. | In orn words, ‘to use the language of the frangwork, 


eo 7 n. ; . ' a, ares 


oe 


. 
~“e. 


components. For. example, Keogh states that ‘éhe real iasue under consideration sen 


¢ 


te, is paphastaing the validity of ther components and the processes ‘that 


° -1 Yr, 


a enter tmto then. Specifically, she enphasizes” the Linkage between asséss- 


ment data and educational dec thioaa’ about LD. children?’ Fs. F 


hal. 7 ; Keogh also shares” her belief, that ‘the’ ‘assessment Bnd identification 


» 
a a. : “8 


- ¢ Sa 


of: learning disabled children, is an important goal, vengevene of Long-term . 
; ana systematit lnvestigation. “Do we, ‘know that this ig ‘rue? How do we 


* , ‘know it ts not a’ waste of toney to assess and montaty these children? 


How. do we know: Eney do not ‘perform more ef fect ively in regular .classroons? 


a, " Do. we have evidence’ ‘thabyintervention are effective? - 4 A 


. aay 
’ : a 


‘ NOE ton ago, I was informally observing a. fifth-grade classroom 

4 ang studying three or four-of. ehe children who did not seem to read well | ae 
P —_ enough “6 keep ” with thats ssons. The oe Mrs.- Diy bointed out 
| \ : 7 one ‘child ho me ane she thought was bright and cohscientious but waa, one \ 


7 "y Of. the lowest readers in the ‘class. Partly. by way of description, and: mo w 


t/~ 


Sauk ly as a request for. advice, abe said, "T “don! t send him to Mr. x (the 
reading specialist for, the. school} | because. E ie he learns more ‘in my | 
classroom." What isa scientifically: ‘based reply. to thie Reacher, I asked 
a “¢ myself ad the blood drained out. of | my ‘face?’ Ina moment this ‘teacher has 
jettisoned the ”refercal-inseagiiAiagnontezcrantnent proceds for children _— 
oe with ‘Learning disabilities because she didn’ t see any, sa eee to- the | 
: ,issching component,” nig] we catr teil her the probability that this 


&. 


ie a , angunt of, that. faprovanent, ghee ts ‘little zabionate for the “practice. v3 


(or the that gation) of. other ebaqgnents re the de| tndon-aaking process. “ee 


. child's centile vip be Amproved by an intervention, and, tell her the ‘ : 


a Po With substantial respect for her judgmant and little ackaptific evidence . 


os a “bo > thd eostrary, r ‘told Mrs. D., ons probably right~-he does isacn 


v® 4 “ . rr ame 


cs, more in. Four BA soe  @ : ‘ | Soom 4 


vo ‘ . ; Sones Poa 4 . a 3 a ” 145 . o a. 
a ) : ba = oS ’ on o ; " ’ : ‘ Bs sa . 


; ; oO G | ‘1% a 
F . - ; oS oe ‘ * . . 
_ ‘In the first are ‘of research discussed by teselayle, thegadequagy _- * 


of “nom: referenfed data for Bcc of. ‘Success tn a highly syetematicl 


oe . instructional ‘Program is being | investigated. Peliseie this study a’ : 


7 , : weighted combination of certain tests will be -identified ‘yhat can, Srediee .- 
those “— learn a great deal from those tho lear: very little from this : 
° - £7 rey . 
teaching program, What, can be done with that formula after it has been, Sg ee © 


devised? One seductive implication that we ‘should avoid is that ‘chiddren 
* . witha profile similar to chee of eacee ordi learners. in that teaching 
program ‘should be Placed Within it. It may be ehet the same profile pre- 


. dicts successful learning in other programs as walt.’ A control study is, | \ 


No v4 ~ 


‘ needed in which the same test data are avatlable for children ihe learn ‘ 


a Brest deal ort learn very little in a less ‘systematic, teaching method. 

: . towever, if there were differences between the Foewila for the. control , 

, DD a group’ in‘ Flexible’ teaching. and the fovea for the eine tuantall group. in 
‘highly systematic each. the. useful result will have béen aaa aa ; a . 

° Teaching will have been validated, he one sense ,. and ar rare aptitude-treat- 


ment ‘interaction at have bade discovered. . . . i 


» if ies : ‘ Pe ; _ Jk :. 
a ‘Reliability of Components arn . = co a me 


) 
2 


ei . Tf a dectision-makirig chain, such‘as the. ,one I have out Lined here, 
4. on 
is to fone ea parece wet each of the components must be reliable. “@® i °* 


~ 


May retabitity, L ‘an Yeferring ‘to coneretency in serforning — function ~* 


in® the: deckason-naking chain. How reliable 4s. ‘the teaching comodaant c 
. 2. te i. - | 
oe = ‘In other words, do: v6 kow how: consistently an gecucersional program of. 


odettata ‘description, say the Distar system taught by an experienced 
. teacher, will meet sotte . agreed ces standatd for’ iq Niactivenens: such as 
° aes year of ‘gain on a standardized comprehensiga test in ten montha of i 
- ¢eaching for’ a given, Population of children? ‘This reliability could | me Pa , 


EMC 3° Gs Se tak ees 


’ . zs ‘a q . 5 wl 
PA FuirToxt Provided by ERIC . ‘ . a 2 ee i a og eee . : ne ' ‘ 
: : : p 


“ ® 
s - 4 a : s a 


a 120 oe <a : [= | : 


stated in the form of the probability mia. any child from Age seven ba 
to fifteen wath normal poonnence on an iweeh Genes test. sueh ‘a's the | 


WLISC and rwo ,OF more years’ deficiency in « _ standardized ‘reading test _ @ 


e * 


é perfortiance will make eu h a gain.. To m knowledge, there are. few data 
é; if 


to. chia eerects and \al though seltescigg eal data has aide, they would 
sie eeictal information: to’ the study of the decision-making procesd, 7 
: "3 ’ % “te: has pe eveP Ey occurred to you that teaching LD children cannot 


Pe le | be validated wi Ehoul being able to pects LD children in sone onaiarent : 


o YS 


‘ ; way. Reliability of decisions in the diagnostic component seems parti- 


’ 


- ey inpBetant 5 ee a atidies of it have been proposed by Yasétayke 


» 
and Lovite during. -thisyconference. In Research Area ce one " study ‘te: 


5) 


es proposed, by Ysseldyke. ‘to examine the” differential diaenbeie of LD ‘and © 


“ton~LD children. “Thar is, 60 professionals wid b6Faghs to sort 120 ee 
> - ar t . "es 
_ cases of indiVidual children ies those who are LD and those’ whajare not vo 
: nO ¢ 
LD,. based on 22 pieces ‘oe ‘teat’ information; jane percentage of cases, “on * 3 


t 


which,” say, mare: than’ 90 percent ef the clfatctans agree tabout this : 


binary decision of LD vs. "non-Lb) will give. some evidedce about relia 
bility of diagnosis into these broad categories, ns 
7 "aS +t 
It seehs to me that ;reldability of: diagnosis tee a " central problem,’ | 
, J 7 ‘ ‘ 


, since most educational placement decisignstare based on 8. Gp c 


%) 


ad 


ildren 


« *. ) 


bags often caddis eres milk and cogkies, and direct, daily measur ent. 
. whereas, non-LD children seldom vacetie any of these., The Seieatied« | 
g 7 auieet lonetca. this problem are: @ Do tliniciang agree, with each orhe 
on ‘the clasgificatton i childten into LD and. non-LD: categories? ‘Designs , 

* such as those presented at this conference may, work to answer this 


(os oo sisatioa: ' (2) Are teams of pvutuesiouale reliable; that is, do they agree ne 


* ek on ae | ig. 


with other teams on the. classification of chtldren? (3) Are clinicians 


or teams of*clinicians consistent over time? That is, do they give the 


f® 
“oe . ” bade nag 


a agme diagnosis on the sane a highly ‘similar cases on two different 


yo, t . ° 


oe occasions? ~ I might add. that I would predict that ‘he: reliability of 


~“ +, 


¢ 


diagnosis witl be inveraely proportional’ to its usefillness for teaching. 
= That is, reliability of LD and non=LD ‘ettenencs may be réasonably high, . 


but thet t value for veachiag is ‘limited. To know that a child has "a 


_ 


teading problem" does not provide a ‘tutor a large: step forward, and the‘ 


‘ oy . 


a - information ould nearly Lae i bé sseuiias by the referral source--the_ 


-o% ’ 


teacher or parent--in any event, ae the other hand, ‘information that the 


ae child needs. to be taught how to divide words. {nto* syllables. for decoding, 
, to expand his or her knowledge of word meanings, particularly in the, area. 


of abstract nouns and verbs, and to expand his or ‘her perception - the 


% 


tiematic’ structure of | stories by agding the detection of "resolution to - 


. his or) her present abilities in aia Beginnings and climaxes. may" = 


» 
o 


yy ‘be Mae eat toa beacher. “However, agreement among, clinicians that these . 
#(or some other) ara the appropriate teactting goals is not likely. to be 
: ~ Ate at this point in the development of the field. | . | ; 

- poe othet investigations presented by neeerer es pgartata to the . 
‘factors. that enter ihte decisions in the gther components in the weci- 


_s . giddimaking Frameyprk’ t t have sketched.” * For example, Research Area 4 bs 


* ehtatls sending queétionnaires to deciaton makers, including srincipale: 
5 : t 
a “special reiaea Nes administrators, and ‘ehild Poevane: demonstration ‘centers. 


The pemponaen=s) will be asked to indicate whic of the ‘components are 


? aS " anoluded di their decision-making Programs, the -particular operations 


‘ 


used, and the peracnnel davoivad, ' Research Area 5 involves studying « 


. 

. . 

. 

y ’ = : 7 e 
; . . - . ai as 
' : ~~, , Ps ae at e- 
a ' : a : . ; ny re 
a i _t ’ 2 : 7 : 
anh i ny 


e P . +3 4 ; r’ 


ERG) EB 


122- « : | | 


fi 
piarenent team Aeticione re Teese cbactvacdaa and tape recording. 


The sdvoud Yesearch area, involving interface with the - computer, will 


study the effect of certain variables, auch as sex, economic eratuel - 


K? 


physicdl attractiveness, and academic vs. social difficulties on diag-. - 
re ym: im $ 2 ae ai i > 
“nostic decisions. muses studiés tryolve i a the. variables that , 


wa influence a diagnosis. ie ae me _ thar a ani could be mye Oth. - 
. | deferring these studies ungila such time #& the agnosis can be vali- 
2. a 
ar. dated. Only: after we ‘enow, whether these‘ educational deciaions make — 
'. a difference cor the teaching of |p children aig which of these educa~ 
onal decisions make a difference. do the decisions themsalyes merit 


° 


‘ inquiry. If a decision is poiuctess and ineffective, theh. the atudy 


. sof how tf 1s made - ie equally pointless “and, inefféctive. 


“Behavior ‘Analybie - -Approach g° Assessment a : a » ©) oe 


‘ ‘The major meats of the’ ‘decision-making process, consisting | 


of referral, assessment, diagnosis, and teaching, are viewed very - 
. ae, | 7 


- differently by the behavior analysis bias incla@ding Deno, Mirkin, 


s ‘ 


and Shinn,- Lovite, Boeing, and others: than they are by the payche 


‘educational studies group ‘of Yaseldyke, Thurow, Keogh, Salvia, and. 


. others. Fhe behavior analysis school claims ‘chat the decision-making 


ros that I outlined peroree of referral from the teacher, dssessment” 
ys \e 
bawertiners, di sfogis ty’ propels | and. teaching’ by a different 


x specialist is ‘inoperative. This decision-making Program 4s dismantled 


qe. 
by peas 8 rendition of Lovitt! 8 ‘view that the teacher must continuously 


y 


evaluate different, techniques whigh are used to solve individual 


- e . . ° nd 


‘problems, and no single instructiona! 


echnique will ever be r : 


“priate for solving all the problem tea by chfldren who. Rave : 


* 


( 


¢ 


» i lty scaghicaly. TascHing is- primarily a matter of trial - : 7 es 
8 


_ Pa a ‘ : a fe 
Ww. a e Or, a hatter of irae and sometimes en a es 
oY rs A 
‘ : of léssons, to cope with changes hat the learning of the child or a group . 
oo™!, ‘ a3 & . 
my , of children. therefore, the behavior analysis group proposes to develop 
a 7 ¢ <= 


‘ eons evaluation systems. which will detect errors’ ‘in the inotructional o 


7 p . Sesh tee quickly. In other words, in the terms I have been using,’ be- . * 
_ ‘cause the teaching component cannot be validated, thes “Feat of the system \ 
does not need to be examined. : 
od 


ya? 
“ ee . 
The behavi> analysis people, on the other hand, also make decisions 


about teaching’ LD children, and their’ decisions appear to he like those 
& ; . 4 
, “nahh, Pot everyone else. The acetatons include: | (1) Referral-~which eonalate eH 


of bie decision to conduct a decision-making routine. This decteton “4a i 


o 


* ‘ &: 
; 4 often made by a teacher based,on his or her own judgments or “pased on 


) Lovitt's Judgment about daily Measurement. (2) Next, . the’ teacher assesses: 


the child” by giving a measurement behavior, for exatiple, a list of =e 


i e as 


20 words that, the teacher would ‘like the child to be able td read orally. 


e « C 
(3)' The teacher will dtaghose the child, which ig to say, whether 


. ® 


the child has ldé&rned or not learned the list according to nore previously 

- 4 ar standard, euch as one Gord oot cesnad correct is 1/10 of a ord in- ee. 
De correct per sean -(4) The inetrye tor engages the deaching ‘component. 
: The ieteuveor decides that jf the child has not earned List A, then | 
Lise’ A wit) be ene, Content, 4f a child hes heasned List Ay Pa 


Ene educational program will “proceed to List Be These decisions, rather 


. a sd e 


common te behavior analysis, dnclade referral, absesgnent, diagnosis, 
° ” — Sn , f 
Y and teaching in their. own way. | There are decisions,within each a iad 


“ 4 


A. 
‘ and -betwedn each Component, and many ope sone may be found. The aepcaanes 


124 


; needed In the behavioral snab oid Evsing is an examination of the 


oo 


be oa e : % 


are occurring at the " tevel of a Yesson in. tie behavior analysis scheme, 


a < ~ 


though they are ‘occurring at the level of the syndrone for the Payche~ 


‘ 7 ry * : Pay oad 
educational | group. \ “ 


Validity and Reliability "" "+ {| Sr a re 


Ny 


- ‘ ‘ . fe . “’ ° 
How are the issues‘ of validity and reliability for the decision-making 


ma 


components addressed by behavior analysis? The walldarton, of teaching is 


° o 


alvays the most difficult problem to confront, and despite? ite _eltsiveness, 
a 


i - attempt to treat it. briefly. The: criterion, méntigned’ evenly, 


was sietiey an educational intervention could be shown to be uniquely 


, iw 
. useful id one subpopulation of children. A more :“anpropt late eriterion 


* 


° 


for the valet of teaching * for beKavior sie regards decisions. 


made about what. to beach and. wheh va it. For example, if a child 


- 
a 


performs: below standard on Word. List A, normally word: ‘List ‘ = taught 


nee ‘it ds learndd to =e (in some sense of the word). if a child | 


e r 


ve not learned Word List ‘A, the teacher seldom goes to bist B for ‘instruc- - 


* ¢ ? 


tion or to Story xX which contains many. words ia Word List A, What- is 


ay 


validity of _this teaching denipion.’ If the long-term goal is for chil’ . 


« 4 o f 


dren to learn Word Lists A, B, C, and D, and. to read Stories X, Y, and 


Z, what is the most ‘appropriate teaching - ‘strategy? rf may be to learn 


- 
e 


+ List A to’ griterion then to read. Story X (which confine words hae List 


: ; . 3 i : t : ; 
A), read list B to criterion, read Story Y, List C and D,yand read Story 


om 4 


Z. But there are other opttonaauck as not learning Word lists but only. 


Stories X, Y, and Z, plus Stories A, B, and C, which also include. the same | 


e va 


vorda, or’ learning A, B,C, D,::Word Lists and Stories X, Y, and 2: to a he 


low oriterion, and then learning them all Again to a high criterion of . 


a 


a proficiency. The history “gf, educditon : is easpiendcat with, eloquent 


_ armchair debate about, ages options. But behavioral analysis can develo 


ae: ; , ‘data regarding. then, *as their advocates are well aware.: 
- The question about validity of dhaghosts essentially amounts: to oe a 
“the, issue of — much learning. (of ova List A or Story X) is needed to § e 


- 


judge. a chi1d as proficient, There is substantial_evidence that Gpakeny - * 


ieaening does not apply to acquisition of word ‘recognition. Children . 


|, learn oe nOEParnae Tules, structure of words, pronunciation of yoko ° 
. * 


words, and other word recognition skills cpiteinuously from first to: + 


twelfth grade. . Mastery oe one word during one ‘lesgon at ‘one age does not 


ca . * 


occur by any reasonably stringent criterion. Although the child, may pro- ‘ 


‘- tMounce a word correctly within a given lesson, his or her Speed, of pro= p 


. 


» nunctation will increase over: the year, his or her knowledge about thé so . 
ae adenlnee in varied contexts will increase, and his. or her ability 


te italia predict: its presence aw a,Glause will improve with gen-. ~ 


. 
. . 


eral language skills.. ‘Both ‘substantial. ante and simple reflection allow 


a 


that we have. all ancreased our knowledge of meanicee of wogde, such as - 
war and peace, or our reading of simple ‘documents, such as: the yeHlow- Se 


Page, And so we are fequited to evaluate the concept of were and 
» 
sequence within the component of diagnosis ‘for applied behavior analyais 


4 ° 
. : bs v 


\, Programs. : or 7 ; . : \: 
The validity of the assessment ann in behavior analysis pro- 


‘Srams leads. to eonenteen that Deno posed as central: what should be 


8 = 


meadured to deterine whether deeaching program is effective? Deno and *© 


his associates indicate that measures they have used or prapose to use’ 


‘Consist of shigh frequen words in isolation or oral reading of words in ° 


4 
Bl 


Pow +. “ gontext. 1 confess to say that ir wag a relief to find out that reading’ a 


an 


* 8 
“tle 


, 
) Since. “the cenit is reading aloud, it appears that :the:behavibr is unger e 3 


. 126 


words, which is. a me probe oe these children, would be assessed 


: : 5 . By 4 - 
dis pene children “yords ‘to read. ' os 


« 


ee 


ate es * THe question I an fateee is w r the task of oral reading is 


* an associa measure in ‘the edu ation 0 -learhing disabled children. 
‘ eS . ™® 


It is elegantly simple to ask #& child to read aloud, count: his errors, 


". 


’ and time his pertormance: The récord 1 sensitive to reinforcement 


contingencies. . Oral reading ne to be # valid dependent variable, 


a 


direct observation. ‘But is it?’ _ Pré@sumably, the feadasental purpose for 


reading is not to make sourids but to derive meaning from Print. In.a 


servien of studies, Lovitt and Eaton ins as ak reading comprehension was 
utterly unrelated to dramatic changes in oral reading rate. and accuracy 


that were wrought Pe clever reinforcehent contingencies. the valsdl®y of °. 


> 


the beoneued assessment deviX o— to be examined, I very much dike. 
the coneepe of validation corps: diat Lovitt proposed. The top of his 


“List ie issues to be validated by the research community is the Lipor- 


2 x 


eines of ‘che response. ‘This reretes to” whether ‘the behvaior being modified 


‘Ns significant. In ,the case,of wore recagnition, the importance ‘is, mod= ts 


-~. 


erate. Reading cannot occur«without this skill, and yet this skill is by, 


-no means suffictent for a atudent to understand ‘a social studids text 


: ; 
: , . 
‘ : ’ 


or to enjoy a bubble gum wrapper. 


Reldabhity | Oe a os ne 


ee 


ah, Wa) 


“The issue of: reliability of, components of rafervai: assessment, ai 


.¢ an 


diagnosis, and teaching in the behvator analysis: approath will not be’ ~~? 


’ BE 


digcwesed now, partly for lack OF time and partly, because behavior 


analysts are squsitive to this caiuas j - a soe ; et 


° 


Cloging : : an ae ae HA | | | 


. 


In elosing,. I would like to refer to an observation by William 


es 5 (NeDermott 71967). He said, i we 3 7 


Our ievioave grows in spots. The Soots may be’ “large or’ 
‘ , gmall, but the knowledge never grows-alliover; some ald 
inowledbe always remains what..it was....Our.minds thus grow 
‘in spots; and like’ grease gpots the, spots spread....The nov =” 
elty qpaks in; it stains the aactent mass. (P. 418- -419) ; sea 


A 


As T, See sit, the research RF oposed ‘at’ this sentennee will ‘provide 


~ “t #8 


a spot of new: inl edgs in the Field of: ‘learning disabilities, and my 
t 
hopes is that it may visibly stain to.a more favorable “color the “ancient 


mass of Literature in this “fleld. . - 4 
i ' a” : 


‘?. 


on 


+ 


te 
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a ar , Summary and: Commentary. . 7 . _ 
; a 44s : , 
@ yf “ John Salvia ‘  & 
. / ane Pennsylvania State University . ; 
= : 3 
ee ; have, two major points that’ I ‘Would Like to make. ‘One is on the, . 


compatibility of one two teams". WORKS, and the second is on the general 
— — 
practicality of the” Institute, | Then, 1 will presént a, Series cs picky 


9 : Pan 


. comments, - and final , some more fo agents oncerns, cays 
: - ‘ae oa rn 7 | 
. ae think that this is one ieee that de divided inte two teams and, 


's . ; a u .4 
: 


to some extent, the two teams' work should ‘be integrated and “sompatible. 
dtm ment{oned four major areas, lof cee ‘aceasadngy classifieation, ” 
7 ies -planning, and re Bi rvstton., 
o _ divided the ‘purposes: of see aces them. I think the division ” 
™.. | no fairly reasonable. Deng" 8 team is concentrating on, in their’ wonte: “Wehe 


a ‘ a . 
. It think -that the two teams have 


h *. 


effectiveness a the appited behavtor analysis ‘approach to assessment in 


programing for aiden with Learning disabilities.": wa would call that 


‘4 . 


_ } a | program ‘planning and program: evaluation. eselayke’ s ‘ae concen- 
trating on ‘the problems of classification, This” seems to me a very reagon~ 


‘ 7 . 


able. diviston of labor. But, -ag I tried “to put the two ‘together: into, 


~ 7 an overall framework, I: found thet the teams ignored ene work of each other 
2 * 
eo “in many areas, and °I ‘think. in one spot, at least, the. work is 3 conttadictory. 


Ba d Like to run through those. I. think the. solution, by the ways is. fairly’ 
‘ ; ad.. " 


ae Simple. The- ws teams need to’sit down and talk abit. — ‘¢ 
: : or ae | a " 
Both | teams’ are, apterested in the. utility of ‘the’ data collected for * wt, 
< . an . ” 
oe a placement and clapei fication, Deno's ‘team devotes: considerable. attention fae op 
A : ca ee 
s _ 7?” “to this Lesue. ‘For example, . they’ address thre issue of person-centered versus -. Te ep 
. hoy a Mg, , . oo ' ~ ra 
, .. situation-centered approaches. to sbecgi. Yaseldyke’ 8 team ie 80 ence? = So 
: Mec oe ° < reas 
. ested ‘tn -asdesement and clessiéication that ehett epeth research ares 1s , oe 
Oi oS a ee ee, oe Z a a Bs eae 2 ; 
a de i) a oe a ig? ° . - 
= - " ‘ , a . a “e? , : cn : ome ' te 
7 : im . _ 
<) ; = y a : : ie 13 G : a : ‘ : H 
E on a S . yo ~ a 
oe : ’ ' ‘ : “4 ‘ F “ @ 


ts . +8 Q . 
eo. 3 oe (see rotated e the identifica on o£ puptth as as learning disabled!’ ety I 


en ; ; 4 ( ; - 
ee a7 woe é : ee 
ae “oy. aio | : “ 7 a = 7 Ps . : . 
a ee Eeologidhr ‘Research on. Team Deetgton Making." Although six stydies = . : 
| ; a “are tSoeed by reablayke"s teda, ¢ the concerns. of Deno! 8 team are. ignored. "Te. the 
ar ee simulatton rasearch of. Yeselayke" team, they eau. but do. not, 


> 


ee Incorporaggionggiong | for. person-centeved dace verayl astuation-centered | data, 


In perspective 12, Déno' 3 team considers por aeda) and ‘social problems as they 
2 


. . : = ou > 7 
> _ » do not slat any mention’ of person ~soctal variables in Yeseldyke' 8 retro~ - 


spective ee tft Research Area One er their sinalation studies ‘in Re- 
search Area Two, The ‘comparison of 1p chiidren with ehiidren who are not 


* labefed, LD but are failing in® school (Ysdeldyke! 8 Research Area Three) 


we 


. “uses the Puers-Hageis Self Concept Scale and. the Peterson-quay, but no 
a) 1 4# . : 
ame direct observations. fin att fairness, oe should ‘be mentioned that the , ‘ 
ele os } 
by * :sahesttontasire studies in Research ‘Area Four. do ask neous personal ~aocial: » 


“ iprtaones I will not talk about Research Area ae now, other than to in 


BY 


. 


w. ae ®2@ 


. eo ee ; 8 
area “icate that ‘although cbaervations of° personal end « sdtial behavior could | a iS 


‘| : 
play ant integral part, maybe ® crucial role, an bias “totLoviig/assesenent, 


» am 


‘this aspect is Sa a Similarly, Deno 8 team has not mpntioned gome of 
the {important iesucw raised by ¥dseLayke' 8. tean. For example, Yeseldyke. . ip 
oi “fevoted,constderpble attention to Gis noton of bias- in assessment. aoe *, 


a meee 
a Yind the issue livesaed by Deno! 8 team only in connection —— situation: 


af - 
ie centered assessment, and hei -z Ud it was dismissed fairly casually. a: 
- o : pi “t son 4 ry fe, , 3 rn 
a a, * Abo. rhe only Menti puna ‘was in the atatement: o. ao 4 ead } 
we oe a ‘ae. sfuringen thidevies with. the teacher w has referred. oy vas 
2 oe ae: »/ Studeht, the rédurce person: detects some las operating with ~~. . ot 
ot =o “ respect’ to. the- teacher's tdentification. of this Particular .° - : aie 


ot child: ab ‘a: . problem; an opportunity. exists to: addreds the teacher's . . 
vat" @ |) bdased expergations, as well ag or eee “Of, the Student’ 8s be- . 


x fie x “Mravion.. ; iviee foe is ee 
_ 8 a} ‘ ee e - : 8 . ; é 
: cee aa addition to. the Lidepature on ‘bianed ‘standgrdi zed ‘ teste and | biased ratings, oe 
= “ao ag . - yl me ee ; | ae ea e 4 seems 7 8 4 7 ee 
r ~ ‘a « : | ¢ ce 7 a = Pe v \, oe _ Pon 
4 . . . eae i e : oN : a ’ to “ nan . \ deo - 7 ~ -_ ‘ 
%, a \ ‘ "4 . o¢€ ° “ee \ ‘ 4 ; ; . 1 
a ‘ bo & - he try ‘ a = & 
| @ - % a ¥ ad ; # ‘~ ¥ I 3) ‘i , ze 4 ; ey — ) 
“ERIC , . : e af ¥ oe : , : \N : 4 3 
. So * ' t } . ee re 
‘ vt ' 4 7 ae ie 


ices. ‘ . ¢ € 


“131 ‘ ee 


‘ne which Yaseléyke, has attended, there ie ‘a small, bulberowing literature — 
: ‘ . : : & te 
‘ on, biased behavioral obfervartons, In fact vane issua oe JABA contains an as 


‘pape by Al Kazdih which talks about problens in behavioral observation; ee 


a 


P ‘ gimple counting is a biasable phénomenon. One. nacticaiar problem that I 
: Su . : 


“have | been eoncetned with a ouaeaver drift. ‘There oe a tendency for ob- 


= 


“servers | to’ chanee their' Mey ce time and not rec the same be 


Neviee. tn an institute in which a substantial ope ica of one team's a 
* 


se wits directed toward bias, some ~ concern ‘with bias should be reflected in the 


. i ‘4 
” ¢ vs @ 
* Notuer team's 3 work. Another biad study with behavioral data is a ’ 


oe study: by Jack Nedegorth: He found ‘that labels: of hyperactivity affected 


e * 1 


counts of out-of~seat, hjperactive kinds of behavior. ; . —_ - cs 
The tase point I wish to make in connection with the.\ ompatibitity of . ps 


the: two teams: is that neither team Proposés: to study the ef ect of the label 


4 ° 


, 2 “learning disailed" on program peAnning Program evaluations or program 


eS implemerttation ' for. eens disabled children. I would think that would 


we 


‘be a ‘very ‘big issye ‘in terms of pias following « assessment, and one ‘that 


could ‘eeatiy be incorporated. “the preceding points have dealt with inte- 2 ; 
: _ on : bs ay 2 4 ‘ * - 
me : gration, pod T: consider them to be merely emiestina Each team has x ae” f 


. 


failed to rake inte account the peEnpactrys or the data i a of che other 


“ ‘ y ‘ 
v 


team. Such omissions cpu jeopardize the validity of varjous experiments. 3 


- ' = 


sa , . think that there may be a | conceptual contradfttion ‘dn the work pro 


peed ee the hve teams. Deno! 8 team is ‘using a ‘highly-structured, behaviorally. 
r} ( 


* 
" 


‘oriented approach to inaerygtion) | They believe ig: wild work. There are 


ty quotes that are {lluserative:” 


. 2 The: primary” f . ‘ie Seopnend: research is to develo Wovdavive 
0s “ever tiein: j es, based on time series data analy#is which, 
een, ae ‘when, ysed: by teachers, improves’ the effectiveness of attempts to. 


‘ modify Rrograna for children. with- -Learning disabilities. 


¢ e - . he od 
n_. : ¢ : . a+ 
. : yo 


and: 


fee ‘ ae 5 ‘ at 
2 cee . . BN ys 2 . ‘ x. 4 H + 
thin - : : 3 x % Ce re at : . ‘“ 3 : -§ , J : 
© eg : 7 es 8 c . oan ap oad : 5 P . & } 
o . % * J so t . . 
t . = . © # . : . ey 3 t 4 


ar Ane - _ The ‘Hesearch activitdes. proposed promige to provide the means 
we * . for improving programd of instruction provided for students 
te e wou sawith’ learning disabilities. 


a : onthe other hand, » Yeqeldyke' 8 team defines as ‘ieaening axeabled | \ 
"ie . f - mer v 

’ only those students wine do not succeed: in such a program, As chey put 

7 i it, a_-syétematic instructional ‘program 
ee, < ‘baked on task aiatyeie ae learner skild development and employing 
[ ee continuous daily measurement...fhas béen highly successful with 

; ', © the majority dF students, but has been unsuccessful with many 
“4 * others. We identify-the former as instructional casualties sand ° 

: ~ "the latter as true learning disabled children, 


a . Now it. seems ‘to me that if’ the true learning disabled chil won't learn 


a iS under the structured acid then Stan has a problem. If, om the éttiee 
* hana, ‘they: do i earii under a systematic: ‘program of: instruction, Shen Jim, , 
a 4 has a problem. And, t think that if chey don' t get together oh it, we 
wo 7 - 2 have a, problen.. Basically the Programs Fite porethers there are not a. 


Reo Om sf 


She gee Lot of problems. or difticubed 8.: Buty, I dp wish that you two would address, . 


. é : ® 
> + . ‘ . 
: : ° \ 
° » fe 


o 2 a those. aaeneee — 
I ‘have some picky podn 8 that, I just can't resist meritioning. I have 7s 


ur. “points for Jim ang one for Stan, but the number is not indicative 
‘ty ‘ 
of the weight. Some of my concerns have been raised earlier, . but T would 

-¢ eX . : 
Du, .» like to — them home. In Research ‘Area Three, Yaseldyke talks apoue _ ° 
p hehe , a 


differential diagnosie of LD and noastD children. - They are going to com= 


2% 

pare’ children who have been diagnosed se lestuing disabled. with children - 

¢% ; 

. . who have pees dimenosed as non-Learning disabled. they are going to. re ae 
“ eS 

re \ : give'a wticdle ries of tests to those ‘120: children, 60, ‘in each group. As 
' a \ {0.4 ste 


' best as I can telly the data on LD subjects are going ‘to be foliected some~ 


: t tthe after the, kids. we been in special education. My question ‘te; How 
an wll — agtrdbute differences . “ to — educarton, sreatipsic, ar. to | oy jee 3 
a _ differences at time, of diaguosie? t think it ia going to be a very difficult | 
io Project to oe ‘* ‘m = of rane, neurological dtudies that, 


4 


“t 


to when that tumor occurred. Was that the same tumor that they found 20 
r) : . Sg 


Si & aa « , ag! 
oF validity. -I think eney do, .I think that tn Your: computer stmt atione, 


. ‘soy ae - a 133 


‘were dota or, people tried to find locations of tumors, waited iveny 


ne 4 
years: and pectormed autopsied. rT: there were no’ fumors, and." ’ 


- 


the fie Recetas there were no tumors, everything was just fine; 


: | . 
if there wag a tumbr on autopsy, one was always open to question as 


N 


es ’ a : : 
years ago?. Perhaps sore fondanegfan is the question of the inter-rater 
agreement of diagnosis of learning disability. Ina follow-up study 


diagnosticians are going ty Los at the records and..differentiate children 
y 


Abe harve been: idenPified as jeanne disabled and not learning disabled. 


f 


My question ts, what Happens if they ‘cannot take a reliable distinction 


n 


when you. go to Compare learning ir scp with, ii children? 


You Have a dichotonous-criter ton Saviaule that by your own research will 


S 
demonstrate no velfability. hie: you do‘not find group iii esc. 


do you attribute 1t to Faulty classification procedures. or to the measures? 


. a 


Similarly; if you do find group differenées, but ic cineianni cannot agree. 


“that one group 1s learning disabled’ and ERE other is: ‘not, as those 
1 og “4 : 


"differences mean? , oe 7 4 «Ss "ae og 


x : ss "i o a F y : \ 


“In Research Area One, you will develop d definition of successful "and 


e 


“-unguccessful students in order to: compare kids who have profited in an LD 


. 4 


peeerem and those who have not. = is not as easy as it appears to make 


. 
g Pa 


a cut An chavs. 0 decide what conat putes nuccesatul and unsuccessful. h 


A numbgr of studies ehet t had something to aa with werd wevieued 


2 


o 


the general ‘conclusion was that these studies. do not offer alot’ of insight . 


in your aueneiotwates eesti and your’ Mecrvarion research, you are 


md 


doing muh the same sorts of ‘things that we did with the attractivendaa @. 


ad We asked teachers what, they thought, we went out to _ oak at what . e 


4 ‘es 24 x : 
é te : - 


, i ‘ be Te ‘ ‘ : ae é . _ an 


tig 


a. 4 C they were doing. in terms “Of sokene: and then we “Lodked at the outcomes. 
. _My last, point for’ tim is that I have some concern about the interactive 
ro : ? ae 
effects in the ‘studies chav: deal with questionnaires, ‘conpater simulation, 
. . : wea observation. ‘As I andevetaud AE they include the same Wayehdleg a: 
- = - 3 wonder’ how mi peeeieinaenig in’ one Study will “atrack neir behavior . « 


Je 


to the next study. I peccetise ‘that. there maces advantages to studviie. 


° 


- - { the same people over time, But tHere a are. ais8 some Secale en “ 
. oo 
‘: The last picky point is-for Stan. I'm réally amazed that he is 
*, going to get ort to deverép a new set of achievement os in all of those 


areas phat are based_ on observational data. I thine "this will combine 


the Probleme ‘of both behavioral observation and standardised testing. 
You are going to: Have to worry about. both intercrater reliability and 
/ regular kinds of reliabilitye i ee S x * 


; ‘ * ~ Practdcality is a more general iasue, that I would like to discuss, 


The question is, how useful is this ar as going to be? That’ is, °- 


ws : 
\ will this Institute produce raneavel at will make|(a ‘difference. in the 


° [. t ‘ 
J * tdentification and treatments of learn ng gteabled pupils? (ry answer is { 
5 a _y wholehearted "maybe." bee ever yening go ell, I guess it coud 


“ " pessimian lies primarify dn the urea’ a decision Making. I think you. ve 


will find a lack of ‘eailin effects, a “lot of error variance, inconsistent 


o gles 


“Judgments; T ‘do mot Shin there’ will be a consistent pattern. My lack 


of confidence tial ‘not stem from doybt pe ‘rhe team's eed to. “do 
' : ! 
cup z ~ research or thg methods that, they will be using, at! 8 just the ‘state 


: ~~ ‘of the art. I “think that we are’ intelledtually, bankrupt ‘ad we are. : ; 


\ CO trying to coe, with ¢ a “systen hat is not very scientific, If we are 4 ; 
oS - Supposed te be a selenetete dtectpline, ‘then we should ne able ‘e agree 
. bas 7 4 ‘ ’ a, | 
ao sas upon what ie ake: observing. In the case of earning dtesbled children¢ ae 


‘ : a 
= we should ‘is _— to agree chat me y> is £ children are\ indéed faarntig ; Fe 


Go ee ee ke os 


cae \ sdigabled. he cannot magne. a group of chemists arguing over whether yew 


hot a sauiiis ontatas fron. i ne a  « 


neeery 
re 


8 i . ~~ 
I disagreg with the notion that we should try to — Set tutttong hy 


issues. I “disagree with the premises on. which - classifisation of 


a , a : Learning stsanffey are based; I disagree with the definitions. This 


doesn’ t leave amuclt The first premise with which I have a great “deal of ra 


4 


8 . ateficuley POAEnNAE mental processes exist as any ine other than descriptions 


_é 


of behavior. Certatnly, they do not exist as ciiges than, can be steevched 


ans formed in the sense that most ere are talking: about and in the 
. 
- _ “sense shag. the defini ton is used. The second premise with which. I have 
; I Hs: 
a*great d¢al af difficulty is that in the’ Joint occurrence of E behaviors, 


\ e 


: : ' process errors: and ¢ errors,. anybody’ can separate a cause from \ 


oY 
. 


f ; : Se > is 
° an. effect on one child. oblem of attribution is key here. The, 


y. . question baked is, what is it ° bat hap caused this child's failure, to 


\ 


pendever” I think we encare in that kind of exercise every time we diag- my 


‘nose a Igarning disabléd ehdld) Deno spoke to that point well in his ™ 


' . w, 


paper. vee to it anges | not maye it go away. - Now, I would not argue © 


clear-cut eines’ The third P emise that I have ¢ a great deal of aif 


‘y 


‘ relfable or cael I see mo reason to believe that scores that do 
-* A . t 


_ hot correlate Perfectly enoutd be the same. “tt! 8 just ‘that simple. pure 


. 


ogee fundamentally, ik reject the notion that, non-educational variables | 
gan be used to classify educational Sires or to provide daar 
the delivery ee educational «programs. As Deno and Mirkin wrote in Data- 
“ . Based Program Modification: = Manual, ‘wiandicapd thenselvew Fernie wp 
ae. - only. tq’ epeculate on why a pup is atteleute to teach. Merely west: 


‘ing a \ handicap often obscures the pupil! s inssructional needs"; <p. 53) oe 


. 


* ‘ ‘ ( ; ; Z Eh . F ‘es a eee “ U 
@ . mi ad : : : : - : . : a - = . , * ” “t oF o ‘ ; i 

ot a pee. {. | - ore eer 
oe 2 a 4 ne, 
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. —_—ea—a_—= 
4 _—« Later on the same save they say; "From our point of view, then, the 


. 
os. 
ea 


; peck ice is never a child"s handicap, it is always the discrepancy between 
desired and actual performance: and progress.' eee ere: many relevant 
oo pupil. veharacteristics, non-esucat tonat vattables, that - ‘cannot be ignored;. 


we know about the effects of hunger. -and ngloutrition, ve know what a: 


€ 


p ©, illness i we _— what ‘uncorrected physical defects do. Yet,. 
ie 


rae 


if we have a Unery ensda, if we haye a ‘child with a séverely Limited “ 


a . Memory ability,” our Job ‘ts Beet to teach the child. | 


;, The aetend problen is with the definition itself. 1 have bea trying 
. le * 
oe _to get @ fix on who- léarning, dieahiea children are, and I would ‘Like to 
‘eo 
7 share. a folksy parspective with you. “Learning disabled children are 
ae 


Pi 

| 4 | . children Wns for no apparent Beason, are pertatming Poorly in’ school. 

ae "No appa ent reason" in that these children. are free ftom debilitating . 
-intellec ual handicaps, enotional problens, . genaory problems, gotok | 


‘ 
handy ees The children are culturally similar to mowe children, but 


' _ something has geet to be causing their poor ‘performance. As 5g see it, ° 1 


- - | here are roughly ae things than can cause’ their poor performance. 

oo . The first phe ig. tests. er ectcteaes caats are’ designed to rank 

4 ' * tnfividuals in terms of ‘relative leVvelk of mastery, to discriminate among 

4 > puphias to separate pupils along various continua. Test .questions chat 
do not jiscriminate among individuals are not included. tn the test. Test. 


questions for which es children | answer bétter than high-scoring 
¢ 


id 
children are deleted. The job 1s to aoeeed the ‘children. Achievement _ 
{ 
tests and intelligence cents, as well as tests of paceiaNay processes 
sg 4 Sg 
that are used to identity learning disabled children are, indadd, ‘norm 


referenced tests. That leaves jus with the conclusion tae owe children 


“Gabe flunk. If we define » norma as 25ch percentile ang nay “toe either 
% 


- . ° 
, és a : . . s . 
. re ee a a a 2% ; Dw. 4 ws 
. . . ; F 
“g YY , : ; : ' Oe i 
5 ; . * * s : : . 
ee . 
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ERIC * De ae Ce Lan. a 


, r ‘ . ‘ yo mer 
[A FuirToxt Provided by ERIC [i 5 . ay by . ? \ 
“ : : : : us ee ao: oe 


( oe ° VW 
an ‘ ' . FY , ; * 
a achievement or/and intelligence, ah we say you Rave to have normal dis 


telligence to be learning disabled and y ‘have to\ have abnormal achieve- 


- r ry 


ment to be teaening disabled. is if we assume tha the tests, are tadésens 


» 


1? 


on dent (wtiteh they are OS cn we will still come laa about 18 pertent 

of fhe | population as meeting pis Wieden of normal {nteligence and 
oor’ achievement in each achievement -area_ tested. The. oint is that. the ~ 
: tests are destoned - ko: identity children who | are behind, Jane if you make a 


- definition based upon low sere ermane, ingtwo aedag. you ne going to have 
Be 


! | children pelling no matter what their internal. state.. Sd tests, I think, a 
can caufe us ,to have a LD ‘population. oe 4 — = 
« i ; =“ rd aad = 
Sdeond, poor teaching. - We don’ t like to admit it, oo. are Roor . 


teachers out there. Bob mith and mace Nereworth ust” che ten" “teaching 


* 


4 disabilities." Children : ith normal ability. may not be cachigying simply 


i a 
° : because: of incompetent tnberuct ton, repeated BY janes beaches. ) 7\ 
oo € 3 iz 
. _The third a an for why pupils are nat doing well tin school 
ate for no: apparent reason is for a teasin that we ~cadinot see - something in? = 


* - 
Mo h 


. side ‘the child. There May be _Somech ing hat’ 8 wrong - with the children, 


scestiine inside.’ t think our social: policy will not allow us td provide 


-compensation for Vietims of poor teaching, or PimeETOpEraty testing, or 


wo who are ‘poorly motivated, but it will provide Boney for children who have 


Y, 


“4 ag matter with: them. If we. are “going eaten the eee 8 , Mi; 
lize 


have probléns ineine and aren’ t making. a in school, thea: to opera 


Sy 


that definition is fairly ‘apie, at: ‘Tedet conceptually. “Somtebody eda ve 
aoe e © tL, 


to teach the childten undef highly *notivating éondtttiond If a child ) 
_ doesn’ t learn, agenn' ‘meet’ sedtqrion within. a ‘specified numbex Of trialé, =. 


; » xv . 
aa ‘we have a funetional peranete of learning disabilities, “uct ee same ‘way - } 
7 : e 7 < ay ry . es 


7 . "3 “ a v4 _ 
ery, _< f a - *., ; : \ : a 
: ’ . . . ‘5 : . oot oe % ig) 
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ot 


. a4 . 
4 . . Pat 


-Yaseldyke indicated. The pcobsseion has not opted fom stich an approach.. 


Moatdefinitions: of ieaeaite disability are scdentifically useless; chey 
a eapty Jargon « or prattle. The. following. is pare of the Federal deus 
‘ eosan "A process deficit or "dyafunction jn one or more of the basic | 
S eayenoroptcal ‘Processes involved in’ understanding or in desing Janguage.” 


~ This is tnotuded 80 that the ‘Pupil is not a victin of poor pa or poor 
aT : : oe 
ro 7 teaching. I: would like to see’ ‘an enumeration of the ati if the 
: 3 
child has.a ae let us list.the Ptodessds that we're conedd- 


-_ 


- 


erins. iiWeceanaesiy he ¢numeration of Sudh proceseas is seldom given 
t ‘ 


| Pecause they depend apes one's theoretical orientation, and the theoretical 
orientation of the person who has/ the power to define has been very impor- * 
tant. But we could talk about i annie need states, Perception; 


perceptual-notor ittegvation, conceptual ta attention, binocular vision, -_ 


& 


; spinal aii coadne. and at Ffduley pumping on a swing. It séems to me that 
3 


if we are going to specify Process, ‘dysfunction, then the peat we can. do. 
is spaciy- the- “process. . a : ‘ 


. Now, back to the fefingeton, The process Dvorder _—s manifest 


a ae in an 1 imperfect abliity to, » cial think, ‘read, habia. spell, or,: 
4 . do mathematical calculations." Think ‘for a ‘moment about imperfect or, 


« 


perfect ability. Have. you’ ever sat at-a 1 Pkesentation and had ee 
wander? Mave: you ever misinterpreted or ; misheard something? Have you. 


: F ever made a logical error or drawn an Anappropriate concfusion from ¢ con~— 


: a. _ fusing data? Have you ever. dearticuléted; spoken. with. an accent, failed 
, . to say what you really meant,? Have you éver been ee rens a paper. and. had 
» = s ; ‘your: mind‘ go faater than your enei1 and left out words? fan you spell ; 


/ ae word in the English Jengauge? Have you ¢ ever made an error in your se 


.s _ checkbook? Bounds like the seven warning stgiaiolot learning ddsabiticy 


‘doesn't 4t?, 0 _  e ee ee oe 


Mee 


Site 
The last part of most : definitions axetades students eligible for 


A 
My 


other kinds | of sperial services. Yet, ! Eien», Chere isa "logical absurdity 


’ - - ‘to the ee that learning disabled children donot include children. who 


'. have iearei te problems which are primarily the result of + -environmental 


a 
oe. . disadvantage." I think that exp in eon fails to take into account severpl 


e . ‘ 


hotisand reports ¢ ‘on the Ampact 3 environment on learning. ~ These ‘and - 


t. : ; 
similar empty phtases ‘lead us {nto ‘a ‘conceptual quagmire. The words. and 
: : ‘ . ond aa : ’ 


/ phrases used to define -learning. disability cannot mean what “they literally 


; . ae They've been given new mye tnae<: "This phenomenon . is generally 


os referred to as _"Humpty Dumpty, "in honor. of. the ‘firet egghead. “Lewis.” 
- a carroll, fa through the Looking Glass, recorded a conversation .hetween - 7 - \ 


‘ 
_ & «. 


. ' . » Humpty Dumpty and Alice. , . ss 
."When tude a wear Humpty Dumpty said in a rather scornful tone, - aa 
"it means just what I choose it to mean, neither more nor less." — ; 
"The question,is," said Alice, "whether you can make words mean _3e 
_ sO many. different things." ."The question is," said Humpty Dumpty, ° - 
oe || (Ut iY “whtctt is tbe master.” 8 7 ee 
If words. in a defini eton can mean anything we want them to | mean, then a 


| surely we’ can. use the definition to ‘ddentity anyeoay we want to ) identity. “4 


Big é 


. ° ‘ies Kirk and Elkins examined . children being served by csiCs, the” aia 
_ facilities for” LD children, they foynd that. ‘the childnen actually erved 
under: the heading of learning disability would. not meet /the current de- 


finition of Learning disability cited in the Federal r&les and regulations. 

One should “not be surprised,” Humpty Dumpty deftteton + result in chtesieay © 
Ss . and capricious pieced this is the. problem Hist Tonelli nd Bene 7 a "4 
“face, if they are relying, upon auch Puocedures,, since they will use LD 


, + : 
e: Ps F 


. populations that ‘are ‘already identified. They' VA gor to make some sort 


of sense out. "of ae eonceptual quagmire, i | 23-2 << eA oe 
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. : ) 
4. ‘ e 7 : 


a7 
+ 


of learning disabled children: ‘requires that inadequate 


2 *, ea 
definitions be opebatsonalined\, We have to do something to turn a defint- 


EiDe date practice. I chant the eibeae thing that we can aay gious current 


"The identification’ 


saremeN devices 1g that they are generally, iestlecues for diagnéstic 
‘ 4 


purposes. But, EoEntak that the time is long aince past for us to be 


— _° 
; so polite. We cannot’ continue to tolerate -such inadequate devices or, the 
+, is ; 


« 


_ continued: création of more inadequate devices. : . ~ 2 * 


. 


és 4 - To conclude, there are some bright! spots when considering ane prac~ 7 


ticality of -the Propased- research. The Deno, team offers a potential AOE . 


\ immediate help in instructional management and development of . IEPs, Te 
¥ 
their work ‘is successful, .I see the likelihood: of cere Antuston Anto 
= - o 
current ‘practices wy behavior ists. They tend to goad , and hey’, tend to ube : 
o~ 


The vaseiigea: tedm can provide ugeful deaediceiane of what 


~ 


. their data. 


{1s really going on in diagnosis and classification. I think they have ee 
» the long-term’ potential to “Provide information which could bee used to” * — 


Eetora current diapicseic and classification pracenges not only for 


oo 


children with learning disabilities but ‘also for children with mild non- ~ 


o 
. 


oa 


‘ cational basis for Seer pupil progress. ‘Frow my own interests ad 


\y own perspectives, I'm satisfied with what th Tnetitute ti goipmto a. 


ae 


oe designed one’ 10" oF "evo additional atudies that. demonstrated first, that, 
9 ‘ : . 

. 38. ‘ _ pupil performance during tiistruction; was .more predictive of subsequent: Sg 
6 a S 


ee diagnosis of. learning disability than were paychonetic, process measures. 2 
Poo «4 ry 


“ 


And second, that. .teachers — pupil performance! data as a basis for in a 


i 
° 


ai ‘ structional decision making vate not biased by haturallytoccurring stimulus” 


1A Fut Toxt Provided by ERIC 


Ate the same time, T would tava bean overjoyed if thease repéardhers a 4 


sensory handicaps. k the Deno; tean also has ‘the longyterm-potential, _ 
. po? Ly . 
ca their learning data prove to be. Val d a efficient, to provide ari édu= : 


“4 r) “ ry : rs 7 
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\ 4% properties for the children. they were teaching. I think that if we —  S 
Sg ; . _ es a . 
ae could have those findings, we could comé out of the swamp. ——- : 
oe ie - , ; ; f * ; ye s ve ‘, 
‘ & é 3 : gaz. * swe : . See: eS ." 
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"HighLigies: “of the Roundtable Discuasion , oe ees 
“eae os 4 Concluding » the Conference ~ —_ a ee 
a "The discussion began with comments: on the Brengine of the two ” vee 


fesearch projects cpesenned by Yeséldykegend Deno. It was ndted ‘ehat 
& . ae ; ' 
- two projects do seem - have different methoiéloniesl ctentactens ; . 


$ a, 


- It’ was suggested that an important approach to blending the rasearch : 


wee 


“would be to get each project ‘to aak the other group 8. quesriche while | OT nes 


Se. 
Fd using their’ own methodology. . For Sram ey "Deno! rs) eamomight ask: what 


‘ 


kinds: of biases occur in dectonen making when, using titie“SampLing daita?.” 


Such a blending of questions vould: provida a typgavf: cross-validation, 


Also, the two, ‘teams ae brainstorm spe Weetants of each other” er 


and possibly: how each would. conduct ‘th eruetes ‘proposed by the othervy, 
F 4 fe 
: team, MetHods, to gome extent, determine. ‘how. -you work ‘at: a problem ‘and 
e ot : 
‘even ‘the’ problems you see, Talking with edith other weaid: Promote the « ; 


te t 


‘sharing of insights! "the suggestion is not that ‘the two teams do each 
a 


" other’ 8 Studies, but rather, ‘that each team aa into actount the im- _ 


‘on 


portant weneeke oe overlap. For eximple, Deno's team should pronett. 
: . ‘ a . \ . 


_ itself, whenever possible, aginst’ biased assessment... And Yeselayke’ 8. 
team should bring ‘the behavicral approach to bear when looking at ie 3 ee 
rh - Py ¢ 7) a ar ; ia 


; speration ot mul tidiseiplinary decision-making team. 


at 
® 


~ 
. 


It was further noted ‘chat it 1s probably a vey healthy radeatce : — ie 

. t , oe 4 
: a 4 ‘ * ‘ 7 
situation ines Pave ether: in ‘an institute do not share a common tethod~ 


ology. The projpcts dn the Inet{tute ‘do ot tat in-4soLation; .the al “4 
Institute setting provides thé “opportudity to: bring individuals together, i hak 


whe can éhalienge one - another's a . dean. ‘The. concern was expressed,  hgvever, 
< as 4§. zi 
that it is “equally important to ‘synthesize the. results of. the itv aw, 7 


. _ proaches, 80 that the ‘various. points of view wile be -hrought together 


* ' ; . e * 


Sen Hy ow dl 


" -1bd | 
oe at some point: Yn time, ea cn oP 4 
2 oe -. s Pm 

A question was then asked about how the two fesearch projects 


’ i4 


ae a dn “the Institute fit, ‘in which teading education, - was’ suggested : “ 


id i”, 


’ : iad 
’ 


that muletple Methodologies be uged Yo approacit the questian of inter- a 


‘ verttion ia reading. nner one > looks cat existing programs that , gre to 


+, 


Mowe. a ¥ _ thoughe rds special, a. common pieces is a high teacher-pupil ratio. : 
| ““Oenerally, descps tions, of: ‘the content or nadine procedure are 80 


“ils as’ ‘to be rather useless. Thus, - to help. learning disabled chil- 


e 


&? 


: aren, we must. discover the niajo baranetets| of thé educational pro- 7 a 
. oe A , . . a an’ 


* rams that help themp. A very. si ple experimental study Yo recometied, 


° r 


e 4 a ‘ « a , 
a one aie would fayolve Seay planned variation studies and an’ “ io 
2 gtbnographi: strategy ta! ‘determine what is going on in programs where > * 4 
oe eo A children’ are’ Successful; . One ould: use a. naturalistic observation s ~ at a. 
¢. fmt 7 . a : if 
o- ” 4 a . be . = ‘ « 


atrategy .to nan out oo Coens in those classrooms where childyen 


‘ . . ° : . . “~ hd ig . a iad eee A 
a < }. Phe mana Lec in reading. - aE you find one a particular in- . 


° - 


ee ae ge ont ‘classroom teacter, oe are in A «position to. develop Bp 
- ‘ | . ee to locate” bchoeace typés of children. so that they | 
: 7 ; ae ean be given. the Serviaes that’ y you focal he’ * doecessful, — them. , tite”. 
Wo ; . might involve looking for cpeljude:trestsent inderacttons tit," oeee’ a ad 


ty 


AP gaag: ~though: oe research using tilat, approach hag ‘been - ‘very tin. 7 r ne 
. 3 
oe 


a =e . to iW commentjng on Ents suggestion, “te was-noted that ATI r 
*' has been grim: not only in the: past, It is ‘particularly distressing. 


va, 


’ e o.. e - 


. a” "when talking ia outcomes. _ Bruce Joyce at Stanford systenatically ca 


had “heachers ‘use ape: ic dastrieciensl approaches, and randomly wovad: 
an / ehted the rough the various approaches. ud was ‘unable to identify + 


es _ one, - instruétional approach that was siteceive any one teacter aeed 


1 ' - . e 
4 : . 4 , > ; 
o 


F a : t ‘& : n- 
” ene ee across MEP FeSChes). and any interactions. , auch pindines suggest . 


eas _ that. judging outcomes of the teaching process {s ‘tremendously complex * 
ar we ¢ ; 
° an@perhaps i tosyncratic,, Furthermore, ,even if one were to identify ppt 
= A . “\ ‘ o os 4 
© aped rudt that pnberactrd with treatments to’ produce specific _Anstpuctdonal 
2 ¢ 


eubeens sy) everything would wash out te the teacher had.a bad aay & 

ee on these points was expressed, but At was ales, eed that 

. ; 2. e 

e the research to date has focused on a rather narrow set of variables. 
‘ es a Tl 


It’ was suggested ees ina sense, sending a as to ‘schogl reflects ‘an 


wa 
, 


7 ATI deciston. Children, whom you: think have an aptitude to; learn are sent ag 
Vv 
_ «£9 schools; others (for example, neapttalieed” children so eee that 
| , their’ aaa of: learning in a classroom are very, low) are not sent “a 
I ar: 
. “oe — ‘school That, is a. decision dine implies an aptitude treatment a 


action. ‘A sence interagtion 1s implied by our decision to 


send. children 


. to schooi at age six. kather than | age two, with the assumption that there a 


a a is more Spectude for jearniae at six than at twof ‘Even finer maa a 
. ‘ \ * 
euch as placing. gtainable youngsters ‘tn one set. of cictivutium goals afd 


a 2 * Stier chiuden tadboter 5 set, involve an ATI paradigm, The assumption 
4 


a ‘of interactions in these cases’ is 5 probably warranted, Sur data have ‘not 
: - 4 6 
, eo ' .. been collected at this level ATL research. has been condueted en finer 
' a aks Aigtaaggs 26 ; aoe 
pac pale te a . 


ee tae ed “variables, such as the: ‘aqteraction of a divergent~thinker teacher 
i ~ ee with ch{ldrén iho are divergent, thinkers. Maybe “- interactions » a 


“ do got exist at that level. But certainly, they do exist at some Yevel..:- 


* -@ 


or we wouldn'’ t have peucer eon ‘8 de “  .g 


C : "Discussion then tusned: to the. potential evsltcabiitey of ‘the e gts _ 


4 eo py 
ry ee . ‘ 


formative evalugeion approach to virtually: any ‘set of: goals de targets, 


‘nig The ‘relevant question then becomes: what arg “the important targets ine, : 


’ - 4 teading,. POeoen Ive developnent, pe enenneee se devetprmer ‘Language, and. 


‘ 


» es 7Z 146 »  % oa 3 7. =. . ° 
ad - 4 : . ee ; * e 
. so on? When these 4 4re acon enaene hee can be empirically validated to - 
o ¥ 


‘determine their value within a formative evaluation system. “Such a - 


_ ae system reqs Kes ‘simple defthitions, of target ‘behaviors; if EneY are 


e . @ rica i. A Bl ° 4 ay 
Ps” 4" ‘complex, the ‘intensive analytic instructional system cannot be applied 
. oe 3 ; 
; ge because there is a ‘lack of consistent agregment, ; 
. we ‘. ; “ * 
™ ‘* The _ importance of auch questions was stregsed. “Tt was noted that 


\e in fate for-example, ‘comprehension ‘should be a target, Some - ‘measures 


— °might deal with ‘reading passages and some might deal with processing at. 
\ ; * ; ¢ be 4 
the sentence level. The ‘credibility ‘of behavior’ analybis, outside of 


° 


“the behavior analysis community, wit be vastly ‘increased _by working on 
problens . that People are concerned with, such as" comprehension, In ive 


‘h thts, it was suggested that apdeifying multiple methods and multiple 


ac outcomes becomes critical, How ee away, are children from their target 


. 


- task? At what point do you get some kind of effect? At nak point does . 


your influence end? ; _ : x 


: The’ quebe ion was peed of why people’ should expect that reaching. a 
' % 


‘lower. level skill, such as word recognition, should affect a higher level 
‘ »? 3 
skill such as comprehension. Comprehension should not be eeected to ip- 


ro 2 


crease when an soneRyventLon program is remediating a lower level Ereease: 
——? -. In reply, it was suggested that a simple-minded approach resulted tn such 


4 Papeeentsenes ‘When a rqading < apuaicia ss failure 1s observed, people 


+ 


\ suenwae that the comprehension failure. is not ourprising because the af 
= pare does not recognize any of the words. ”. ‘$0, the simple approach’ 
‘ta to remediate the word recoghition problems ¥o increase comprehension. 


. - The correct approach probably pike téach sah but thaé is ° 


€ 
vA ng ced , ‘ » : 


di fftcule. because we & not men what is involved. Some people have 


suggested that mikcuatilepds eecesuaey to comprehension, and this seems 


‘clear. . ‘ - ni 


ih curriculum areas, outside of what da? wauene specifically in ithe 


during: a year; that ae one tenth of a point change each 


‘day. Such: a. channt could not Be measured reliably sveey day.  paviatie i : 


ve the study surveying: assessment ante 


« i : . td 5 
year success, by looking at their predictive value after two ye > A 
and also by seeing whether they- ‘Predict aa with a secon group, 3 ma 
of children. Second, the study Bhould be conducted for more one’ * - 
z : . _ 4 


systematic instructional program. 
~~ It eae kee: however, -that aut results will provide valuable 


information even if it is possible to dgnonstrate in: oni Mone Siguatica 


to be true*up to a certain level. But, othe. area te "stilt yutte. tne ‘ 


’ he -. 4 5 "aay 
aaa Paes 


Commefits were also made on the possibility Of measuring changes. = 


vd 


classtoom. It was pisueeced that: it tay ‘be niathematically impossible. a a : 


A 


ta. do that because you would only expe ect 10 té 20 points. of —, eae . 


the only alternative .is to track:changes in what is being ee : i = 
specifically in the program. yin addition there” seens to be the poten~, ? = 
tial for,a higher rate of change the simpter the task being. measured. 
Thus, we. find reladively hist zates of change for word rgcognition, ; 


but not for comprehension, nowever compyhension is def mass 


eee then turned to. some cautions that should be observed 


ents to determine chose that 


predict success in a systematic structional progkan. First, it. might =, = 


be best to crossvalidate the fasures discovered’ for predicting one- 


¢ 
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* that certain meagures ‘do predict success. The most: important con- | 


tribution of the study will be in tts tncercetfhsonsnips with the other Pg 
utudies-being: conductad at the Institute. The vanutes fu be lookd Ba 
uP. 


at gn context ‘and asa pare of an: integrated fesearch endeavor. oS a 


ASE 


= 


~< 


- approach night be thai phese educators could generate related research 


ue 


_ school personnel are "under oP a to: develop bro ograms and - ‘services 


; "4 
in the ddraction ‘a classroom: ‘relevancy by holding sd Roundtable > 


Conference with siueatere in attendance. as Roundtable wanbers and ao ' 


. Such efforts, perhaps by allowing aiucdtets to be’ involved in. the 


g 
~probrena, Participant observers who are employed in school systems 


‘results obtained would’ have enpect application in Py Tanaroons eed i Fie 


——Highlights ‘of the’ ‘Open Biscudeica: 


Z 7 Concluding the Conference ° . A 
* { ¢ s . - . . 4 
*- Dtheugsion began with. comment's and questions regarding the 
extént to which each of the is federally funded | Institutes for : 4 


Po 


2 
» e ae 
Research on Learning Diskbilities is ee research on practical : 


s 
indicated that practical | problems must be addressed and solved now; 


e a 


e . 
today, programs and services that. are beneficial to learning disabled’ DoS 


° - 
: 


é : PN 
cha ldres, 3 : a oo ‘ 


~ 


“While considerable pressure is generated eh instant fe solution of 
complex enduring probfens, it was agreed. thaé@ practitioners should not 
rely on ‘the results of inadedaiee and inconclusive research in ebeaae . 
lating services and pre nee ‘Roualietecesea way che ct that © : 
4 


research should be’ dtyectet toward solution of ‘practical sorab lens and 


issues, should not be isolated, and abpuld bd ak that knowledge ad 


sthool environmente. , 


“4 


_” The Institute was commendged ' for the inftial step it has. taken 
i %y . 


partdctpant-observers, The Institute was encouraged to continue 


e 
» 


Institute as visiting observers. An important: spin-off of thiggm 


. 


in their own sthool systems. | It was alao suggested thet the Institute 


e 
Cn 
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ment work with’ educators” to détermine or ‘density posetone appropriate, 


précedures, sevhane aaa on yatues of educators and caguavchaue, 
before all data are’ in and analyzed. “This suggestion, wag made as a 7 


ba . 


possible. apgroach to che clear need FOr aes ‘epucational policies, 


¢ - 


especially as they relate to. ‘the placgnent ban slecision-making pro~ > 
cedure.” oo - _ “a 


: : . , . © —— 
A distinction was: ‘proposed ha ats applied nesesych and practical 
7 ) 


peer een Afthough applied ‘research way ty wad th it the possibility 


é 


ad 


of betug practically applicdhle im echool settings, current practical 


cassareN 


* 
issues involve such things as alia: puftic Law 94-142 . 


° So 


ov 


and _confogming to. existing yr on the evaluation of children , 


with spec#fic Leatning disabilitie « These practical Vagied are urgent 


% 
and’ \paltenging. The paradigm ‘of conduct ing research on problems 


perceived by the schools, rather than those generated ‘by tesearchers, : 


was qapporeee: “tt was “suggested that the questions being mnves tagated 


by the ui may not be those of teachers, parents, “and children in’ 


the equeattonat system. A Pel@vant reséarch issue, for example, might - 


tnvolve devs apie and evaluating various assessment models ,, incfuding _ 


soa 


“che thodel enbodted in Bhé federal regulations, as well as ‘ other models 


‘ 


‘that’ the researchers feel might be of erence ca a It was suggested | 


that a "consumer approach" is needed; research should ta e dete account. 
P 


e as 


, the. consumer acceptabili dy of procedures that. are to be recogmended on on. 
‘the basis of research data;.,. Preferences, rather than data, often: 

_ determine wh rocedures will be implemented 7 the field. a 

In response to these comment s,*4t ‘was re chat the probfem 


= 


may not be so much in rhe seirearch wit saad as in the tatlure 


o s x 

e : o 
_* re ao Aa , 
- , ¢ 


, ? ro 
ae ; . ae 
} fs, ' : - 1 e yo : 
as ; La = ye a , 
pce . ' n a - 


ve 


« 


< ., { . t , t 
ae o : + ‘ - oe . : = : , eh - 7 é ta = : 
‘ i a A c- : : , “ey ‘- . ; os . : ae a 
4 . “ee ' , * 4 ” . . . 
7 fo . : ‘ _ ee 8 
F ys - ° . . " % e ‘ow aa : ‘ y oo 
= - of wesearchars to be in contdct with Heople in the schools. ,This 
; 4 $ 
7 "4 doep not mean that researchers should neceasarily formulate their al 
i ad relaareh questions solely sd talking to educators. An example of 
| why this ia the case comes from the area of behaviorism. -Behavioral 
P SS - - ‘ 
" 7 research had virtually nothing to do wath the practical world for 


— nearly 30 oe years, yet te terms of practical applicability today, 
, tt is probably. the shining. glory of ‘psychological regearth, But, in. 


order to Kadena practically applicable, the researchers had to talk 


e. 


to educators. . , _ tf 
(. It was } then noted that there, are literally thousands of” Tesearch~ 
. able Roaies ‘in the ais of learnihg disabilities, most Sf them ¥ 


‘tremendously complext Decisions have to be made about a apacttic: 
number of questions to ‘ask; such decisions are fakinienwed by. parents, 
: aavbente agencies, eaweatehoes: ead a variety of other sources. At 


this. point, people have to aacept the fact that only a limited set of 
i 


e 


‘tssues ‘can be researched .at any one time. esious is ChOARDY saying . 


that ‘the research should be practical. Popup related to assessment \ 


f 
_ and dgeipion making. are ertticat yn education scodaye : 


/ , ~ " ‘The Sisctwaton ‘concluded with comments on the need to keep 


Cad 2 


woe 


‘communication lines open in all dfirections--to and ftom educators ‘in 


‘the Figla and to and from other researchers. The comment was made 


Bg noire atten teatemabe ah + ~ 


that the Institute was formed to conduct ectenti fic research, Such 


: npeeeneen must be publicly dhaatvable with rapdieaite ouccomes In "8 
‘ so8 ‘ ; r , 
oat obder to be educationally valuable, the first consideration must, be 
; . « 
_ that the research is a scientific enterprise. 
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